Skip to content

C
ContenT
Commit 31bfe20c authored by Roman Alifanov's avatar Roman Alifanov
Browse files
Options

Rewrite compiler: IR-based architecture replacing bootstrap

Replace the bootstrap mixin-based compiler with a new IR-based pipeline: Source → Lexer → Parser → Resolver → TypeChecker → IR Builder → Optimizer → Backend New architecture (compiler/): - semantics/: scope resolution, type system, type checker - ir/: intermediate representation with stable node IDs - optimizer/: DCE via call graph BFS, CSE, constant folding - backend/bash/: bash code generation from IR - backend/awk/: AWK code generation sharing same IR - symbols/: symbol table with LSP-ready serialization Key improvements: - Explicit shell command detection (IRCall.is_shell_cmd) - Namespace prefixing only in bash backend, not in resolver - DCE reduced from 580-line fixed-point to ~40-line BFS - CSE uses stable node_id instead of Python id() - Shell commands in assignments use || true for set -e safety - No local keyword in global scope (when/foreach) - Coproc uses exec to prevent orphan child processes All 400 tests pass. Verified with real-world unified-theme-switcher service.
parent 5f7339cd
Show whitespace changes
Inline Side-by-side
Showing with 9462 additions and 8397 deletions
README.md
View file @ 31bfe20c
...@@ -38,7 +38,7 @@ git clone https://gitlab.eterfund.ru/ximperlinux/ContenT.git ...@@ -38,7 +38,7 @@ git clone https://gitlab.eterfund.ru/ximperlinux/ContenT.git
cd content cd content
``` ```
Requires Python 3.8+ (bootstrap compiler). Requires Python 3.8+.
### System-wide (Meson) ### System-wide (Meson)
...@@ -50,7 +50,7 @@ sudo meson install -C builddir ...@@ -50,7 +50,7 @@ sudo meson install -C builddir
Installs: Installs:
- `/usr/bin/content` — CLI entry point - `/usr/bin/content` — CLI entry point
- `/usr/share/content/bootstrap/` — compiler (Python) - `/usr/share/content/compiler/` — compiler (Python)
- `/usr/share/content/lib/cli.ct` — standard library - `/usr/share/content/lib/cli.ct` — standard library
- `/usr/share/content/lib/cli.sh` — precompiled CLI library - `/usr/share/content/lib/cli.sh` — precompiled CLI library
...@@ -572,58 +572,6 @@ Features: ...@@ -572,58 +572,6 @@ Features:
- [Language Specification](LANGUAGE_SPEC.md) - [Language Specification](LANGUAGE_SPEC.md)
## Project Structure
```
bootstrap/ # Bootstrap compiler (Python)
├── main.py # CLI entry point
├── lexer.py # Tokenizer
├── tokens.py # Token type definitions
├── parser.py # Recursive descent parser, AST generation
├── ast_nodes.py # AST node classes
├── errors.py # Error handling
├── constants.py # Codegen constants (RET_VAR, TMP_PREFIX, etc.)
├── methods/ # Unified method registry (bash + awk)
│ ├── base.py # Method dataclass
│ ├── string.py # String methods
│ ├── array.py # Array methods
│ ├── dict.py # Dict methods
│ └── ... # http, fs, json, logger, math, time, process_handle, etc.
├── dce.py # Dead code elimination
├── codegen.py # Main Bash code generator (mixin coordinator)
├── expr_codegen.py # Expression generation (mixin)
├── stmt_codegen.py # Statement generation (mixin)
├── class_codegen.py # Class/method generation (mixin)
├── dispatch_codegen.py # Method dispatch, assignments (mixin)
├── decorator_codegen.py # Decorator wrappers (mixin)
├── cse_codegen.py # Common subexpression elimination (mixin)
├── stdlib.py # Standard library generation (mixin)
└── awk_codegen.py # AWK generator for @awk (mixin)
lib/ # ContenT libraries
└── cli.ct # CLI library (urfave/cli style)
tests/ # Test suite
├── helpers.py # Shared test helpers (run_ct, compile_ct)
├── test_lexer.py # Lexer tests
├── test_parser.py # Parser tests
├── test_basics.py # Basic tests (print, variables, arithmetic, loops)
├── test_functions.py # Functions, lambdas, callbacks, parameter passing
├── test_classes.py # Classes, objects, field assignment
├── test_methods.py # String/array/dict methods, method validation
├── test_stdlib.py # Standard library (env, json, fs, with)
├── test_decorators.py # Decorators, typing, @test, user decorators
├── test_awk.py # AWK functions (map/filter, sync, assert)
├── test_shell.py # Shell commands, pipes, mixed pipes
├── test_async.py # Background processes (async/await/on, pid)
├── test_namespace.py # Namespace/using tests
├── test_busing.py # Bash library import tests
├── test_build_lib.py # Library build tests
└── test_autoscan.py # Auto-scan tests
examples/ # Example .ct programs
```
## License ## License
[AGPL-3.0](LICENSE) [AGPL-3.0](LICENSE)
README_ru.md
View file @ 31bfe20c
...@@ -38,7 +38,7 @@ git clone https://gitlab.eterfund.ru/ximperlinux/ContenT.git ...@@ -38,7 +38,7 @@ git clone https://gitlab.eterfund.ru/ximperlinux/ContenT.git
cd content cd content
``` ```
Требуется Python 3.8+ (bootstrap-компилятор). Требуется Python 3.8+.
### Системная установка (Meson) ### Системная установка (Meson)
...@@ -50,7 +50,7 @@ sudo meson install -C builddir ...@@ -50,7 +50,7 @@ sudo meson install -C builddir
Устанавливается: Устанавливается:
- `/usr/bin/content` — CLI точка входа - `/usr/bin/content` — CLI точка входа
- `/usr/share/content/bootstrap/` — компилятор (Python) - `/usr/share/content/compiler/` — компилятор (Python)
- `/usr/share/content/lib/cli.ct` — стандартная библиотека - `/usr/share/content/lib/cli.ct` — стандартная библиотека
- `/usr/share/content/lib/cli.sh` — прекомпилированная CLI-библиотека - `/usr/share/content/lib/cli.sh` — прекомпилированная CLI-библиотека
...@@ -566,58 +566,6 @@ python3 content run examples/telegram_echobot/telegram.ct examples/telegram_echo ...@@ -566,58 +566,6 @@ python3 content run examples/telegram_echobot/telegram.ct examples/telegram_echo
- [Спецификация языка](LANGUAGE_SPEC.md) - [Спецификация языка](LANGUAGE_SPEC.md)
## Структура проекта
```
bootstrap/ # Bootstrap-компилятор (Python)
├── main.py # CLI точка входа
├── lexer.py # Токенизатор
├── tokens.py # Определения типов токенов
├── parser.py # Рекурсивный спуск, генерация AST
├── ast_nodes.py # Классы узлов AST
├── errors.py # Обработка ошибок
├── constants.py # Константы кодогенерации (RET_VAR, TMP_PREFIX, etc.)
├── methods/ # Единый реестр методов (bash + awk)
│ ├── base.py # Method dataclass
│ ├── string.py # Строковые методы
│ ├── array.py # Методы массивов
│ ├── dict.py # Методы словарей
│ └── ... # http, fs, json, logger, math, time, process_handle, etc.
├── dce.py # Устранение мёртвого кода
├── codegen.py # Основной генератор Bash-кода (координатор миксинов)
├── expr_codegen.py # Генерация выражений (миксин)
├── stmt_codegen.py # Генерация statements (миксин)
├── class_codegen.py # Генерация классов/методов (миксин)
├── dispatch_codegen.py # Диспатч методов, присваивания (миксин)
├── decorator_codegen.py # Обёртки декораторов (миксин)
├── cse_codegen.py # Устранение общих подвыражений (миксин)
├── stdlib.py # Генерация стандартной библиотеки (миксин)
└── awk_codegen.py # AWK-генератор для @awk (миксин)
lib/ # Библиотеки на ContenT
└── cli.ct # CLI-библиотека (стиль urfave/cli)
tests/ # Тестовый набор
├── helpers.py # Общие функции тестов (run_ct, compile_ct)
├── test_lexer.py # Тесты лексера
├── test_parser.py # Тесты парсера
├── test_basics.py # Базовые тесты (print, переменные, арифметика)
├── test_functions.py # Функции, лямбды, колбеки
├── test_classes.py # Классы, объекты, присваивание полей
├── test_methods.py # Методы строк/массивов/словарей
├── test_stdlib.py # Стандартная библиотека (env, json, fs, with)
├── test_decorators.py # Декораторы, типизация, @test
├── test_awk.py # AWK-функции
├── test_shell.py # Shell-команды, pipe
├── test_async.py # Фоновые процессы (async/await/on, pid)
├── test_namespace.py # Тесты namespace/using
├── test_busing.py # Тесты импорта bash-библиотек
├── test_build_lib.py # Тесты сборки библиотек
└── test_autoscan.py # Тесты авто-скана
examples/ # Примеры .ct программ
```
## Лицензия ## Лицензия
[AGPL-3.0](LICENSE) [AGPL-3.0](LICENSE)
bootstrap/__init__.py deleted 100644 → 0
View file @ 5f7339cd
__version__ = "0.1.0"
bootstrap/ast_nodes.py deleted 100644 → 0
View file @ 5f7339cd
from dataclasses import dataclass, field
from typing import List, Optional, Any, Union
@dataclass
class SourceLocation:
line: int
column: int
filename: str = "<stdin>"
@dataclass
class ASTNode:
pass
@dataclass
class TypeAnnotation (ASTNode):
name: str = ""
is_array: bool = False
element_type: Optional['TypeAnnotation'] = None
key_type: Optional['TypeAnnotation'] = None
value_type: Optional['TypeAnnotation'] = None
param_types: List['TypeAnnotation'] = field (default_factory=list)
return_type: Optional['TypeAnnotation'] = None
location: Optional[SourceLocation] = None
@dataclass
class Expression (ASTNode):
pass
@dataclass
class IntegerLiteral (Expression):
value: int = 0
location: Optional[SourceLocation] = None
@dataclass
class FloatLiteral (Expression):
value: float = 0.0
location: Optional[SourceLocation] = None
@dataclass
class StringLiteral (Expression):
value: str = ""
has_interpolation: bool = False
location: Optional[SourceLocation] = None
@dataclass
class BoolLiteral (Expression):
value: bool = False
location: Optional[SourceLocation] = None
@dataclass
class NilLiteral (Expression):
location: Optional[SourceLocation] = None
@dataclass
class Identifier (Expression):
name: str = ""
location: Optional[SourceLocation] = None
@dataclass
class ArrayLiteral (Expression):
elements: List[Expression] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class DictLiteral (Expression):
pairs: List[tuple] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class BinaryOp (Expression):
left: Optional[Expression] = None
operator: str = ""
right: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class UnaryOp (Expression):
operator: str = ""
operand: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class CallExpr (Expression):
callee: Optional[Expression] = None
arguments: List[Expression] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class MemberAccess (Expression):
object: Optional[Expression] = None
member: str = ""
location: Optional[SourceLocation] = None
@dataclass
class IndexAccess (Expression):
object: Optional[Expression] = None
index: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class Lambda (Expression):
params: List[str] = field (default_factory=list)
body: Union['Block', Expression, None] = None
location: Optional[SourceLocation] = None
@dataclass
class ThisExpr (Expression):
location: Optional[SourceLocation] = None
@dataclass
class BaseCall (Expression):
arguments: List[Expression] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class NewExpr (Expression):
class_name: str = ""
arguments: List[Expression] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class AsyncExpr (Expression):
expression: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class Statement (ASTNode):
pass
@dataclass
class Block (Statement):
statements: List[Statement] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class ExpressionStmt (Statement):
expression: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class Assignment (Statement):
target: Optional[Expression] = None
type_annotation: Optional[TypeAnnotation] = None
operator: str = "="
value: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class ReturnStmt (Statement):
value: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class BreakStmt (Statement):
location: Optional[SourceLocation] = None
@dataclass
class ContinueStmt (Statement):
location: Optional[SourceLocation] = None
@dataclass
class IfStmt (Statement):
condition: Optional[Expression] = None
then_branch: Optional[Block] = None
elif_branches: List[tuple] = field (default_factory=list)
else_branch: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class WhileStmt (Statement):
condition: Optional[Expression] = None
body: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class ForStmt (Statement):
variable: str = ""
iterable: Optional[Expression] = None
body: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class ForeachStmt (Statement):
variables: List[str] = field (default_factory=list)
iterable: Optional[Expression] = None
body: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class WithStmt (Statement):
variables: List[str] = field (default_factory=list)
resources: List[Expression] = field (default_factory=list)
body: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class TryStmt (Statement):
try_block: Optional[Block] = None
except_clauses: List[tuple] = field (default_factory=list)
finally_block: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class ThrowStmt (Statement):
expression: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class DeferStmt (Statement):
expression: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class NamespaceDecl (Statement):
name: str = ""
statements: List[Union[Statement, 'Declaration']] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class UsingStmt (Statement):
namespace: str = ""
alias: Optional[str] = None
names: Optional[List[str]] = None
location: Optional[SourceLocation] = None
@dataclass
class BusingStmt (Statement):
name: Optional[str] = None
path: str = ""
location: Optional[SourceLocation] = None
@dataclass
class AwaitStmt (Statement):
expression: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class OnSignalStmt (Statement):
signal: str = ""
body: Optional['Block'] = None
location: Optional[SourceLocation] = None
@dataclass
class RangePattern (Expression):
"""Range pattern for when branches: 1..10"""
start: Optional[Expression] = None
end: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class WhenBranch:
"""Single branch of a when statement"""
patterns: List[Expression] = field (default_factory=list) # values, ranges, or 'else'
is_else: bool = False
body: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class WhenStmt (Statement):
"""When statement (pattern matching)"""
value: Optional[Expression] = None
branches: List[WhenBranch] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class Declaration (ASTNode):
pass
@dataclass
class Parameter:
name: str = ""
type_annotation: Optional[TypeAnnotation] = None
default: Optional[Expression] = None
is_variadic: bool = False
@dataclass
class Decorator:
name: str = ""
arguments: List[tuple] = field (default_factory=list)
object: Optional[str] = None
location: Optional[SourceLocation] = None
@dataclass
class FunctionDecl (Declaration):
name: str = ""
params: List[Parameter] = field (default_factory=list)
return_type: Optional[TypeAnnotation] = None
body: Optional[Block] = None
decorators: List[Decorator] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class ClassField:
name: str = ""
type_annotation: Optional[TypeAnnotation] = None
default: Optional[Expression] = None
location: Optional[SourceLocation] = None
@dataclass
class ClassDecl (Declaration):
name: str = ""
parent: Optional[str] = None
fields: List[ClassField] = field (default_factory=list)
constructor: Optional['ConstructorDecl'] = None
methods: List[FunctionDecl] = field (default_factory=list)
location: Optional[SourceLocation] = None
@dataclass
class ConstructorDecl (Declaration):
params: List[Parameter] = field (default_factory=list)
body: Optional[Block] = None
location: Optional[SourceLocation] = None
@dataclass
class Program (ASTNode):
statements: List[Union[Statement, Declaration]] = field (default_factory=list)
location: Optional[SourceLocation] = None
bootstrap/awk_codegen.py deleted 100644 → 0
View file @ 5f7339cd
import re
from typing import List
from .ast_nodes import (
FunctionDecl, Assignment, ReturnStmt, IfStmt, WhileStmt,
ForStmt, ForeachStmt, ExpressionStmt, BreakStmt, ContinueStmt,
WhenStmt, RangePattern, Lambda, Decorator,
Block, Identifier, IntegerLiteral, FloatLiteral, StringLiteral,
BoolLiteral, NilLiteral, ArrayLiteral, DictLiteral, BinaryOp,
UnaryOp, CallExpr, IndexAccess, MemberAccess
)
from .methods import get_awk_builtin, generate_awk, MATH_METHODS, TIME_METHODS
from .constants import RET_VAR
class AwkCodegenMixin:
"""Mixin class for AWK code generation.
This mixin provides methods for compiling @awk decorated functions
to AWK instead of Bash, giving ~2000x performance boost on
string/numeric operations.
"""
def _awk_scan_types (self, stmts: list) -> dict:
"""Scan statements and determine variable types from assignments."""
var_types = {} # var_name -> "array" | "dict" | "string"
for stmt in stmts:
if isinstance (stmt, Assignment):
if isinstance (stmt.target, Identifier):
var_name = stmt.target.name
if isinstance (stmt.value, ArrayLiteral):
var_types[var_name] = "array"
elif isinstance (stmt.value, DictLiteral):
var_types[var_name] = "dict"
elif isinstance (stmt.value, StringLiteral):
var_types[var_name] = "string"
elif isinstance (stmt, IfStmt):
var_types.update (self._awk_scan_types (stmt.then_branch.statements))
for _, elif_block in stmt.elif_branches:
var_types.update (self._awk_scan_types (elif_block.statements))
if stmt.else_branch and isinstance (stmt.else_branch, Block):
var_types.update (self._awk_scan_types (stmt.else_branch.statements))
elif isinstance (stmt, (WhileStmt, ForStmt)):
var_types.update (self._awk_scan_types (stmt.body.statements))
elif isinstance (stmt, ForeachStmt):
var_types.update (self._awk_scan_types (stmt.body.statements))
elif isinstance (stmt, FunctionDecl):
var_types.update (self._awk_scan_types (stmt.body.statements))
elif isinstance (stmt, Block):
var_types.update (self._awk_scan_types (stmt.statements))
return var_types
def _awk_escape (self, s: str) -> str:
"""Escape string for embedding in bash single quotes."""
return s.replace ("'", "'\"'\"'")
def _awk_make_emitter (self):
"""Create AWK code emitter with indentation tracking."""
lines = []
indent = [0]
def emit (line): lines.append (" " * indent[0] + line)
def inc (): indent[0] += 1
def dec (): indent[0] -= 1
return lines, emit, inc, dec
def generate_awk_function (self, func: FunctionDecl):
"""Generate a function that runs as inline AWK instead of Bash."""
name = func.name
self.emit (f"{name} () {{")
with self.indented():
self._generate_awk_function_body(func)
self.emit ("}")
self.emit ()
def _generate_awk_function_body (self, func: FunctionDecl):
"""Generate the body of an AWK function."""
validate_decorator = None
for dec in func.decorators:
if dec.name == "validate":
validate_decorator = dec
break
nested_funcs = []
main_stmts = []
for stmt in func.body.statements:
if isinstance (stmt, FunctionDecl):
nested_funcs.append (stmt)
else:
main_stmts.append (stmt)
self._awk_var_types = self._awk_scan_types (func.body.statements)
self._awk_validate = validate_decorator
input_foreach = None
input_idx = -1
for idx, stmt in enumerate (main_stmts):
if isinstance (stmt, ForeachStmt):
if isinstance (stmt.iterable, Identifier) and stmt.iterable.name == "__input__":
input_foreach = stmt
input_idx = idx
break
if input_foreach:
params_v = [f'-v {p.name}="${{{i + 2}}}"' for i, p in enumerate (func.params[1:])]
else:
params_v = [f'-v {p.name}="${{{i + 1}}}"' for i, p in enumerate (func.params)]
params_str = " ".join (params_v)
awk_cmd = f'"$__ct_awk_cmd" {params_str}'.strip () if params_str else '"$__ct_awk_cmd"'
if input_foreach:
before_stmts = main_stmts[:input_idx]
after_stmts = main_stmts[input_idx + 1:]
begin_lines, begin_emit, begin_inc, begin_dec = self._awk_make_emitter ()
if self._awk_validate:
self._awk_emit_validation (func.params, begin_emit)
for stmt in before_stmts:
self._awk_stmt (stmt, begin_emit, begin_inc, begin_dec)
line_lines, line_emit, line_inc, line_dec = self._awk_make_emitter ()
if len (input_foreach.variables) > 1:
line_emit (f"{input_foreach.variables[0]} = NR")
line_emit (f"{input_foreach.variables[1]} = $0")
else:
line_emit (f"{input_foreach.variables[0]} = $0")
for stmt in input_foreach.body.statements:
self._awk_stmt (stmt, line_emit, line_inc, line_dec)
end_lines, end_emit, end_inc, end_dec = self._awk_make_emitter ()
for stmt in after_stmts:
self._awk_stmt (stmt, end_emit, end_inc, end_dec)
self.emit (f"{RET_VAR}=$({awk_cmd} '")
for nf in nested_funcs:
self._awk_helper_func (nf)
if begin_lines:
self.emit ("BEGIN {")
for line in begin_lines:
self.emit (f" {self._awk_escape (line)}")
self.emit ("}")
self.emit ("{")
for line in line_lines:
self.emit (f" {self._awk_escape (line)}")
self.emit ("}")
if end_lines:
self.emit ("END {")
for line in end_lines:
self.emit (f" {self._awk_escape (line)}")
self.emit ("}")
self.emit ("' \"$1\")")
else:
awk_lines, awk_emit, awk_inc, awk_dec = self._awk_make_emitter ()
if self._awk_validate:
self._awk_emit_validation (func.params, awk_emit)
for stmt in main_stmts:
self._awk_stmt (stmt, awk_emit, awk_inc, awk_dec)
self.emit (f"{RET_VAR}=$({awk_cmd} '")
for nf in nested_funcs:
self._awk_helper_func (nf)
self.emit ("BEGIN {")
for line in awk_lines:
self.emit (f" {self._awk_escape (line)}")
self.emit ("}')")
self.emit ('local __awk_rc=$?')
self.emit (f'echo "${{{RET_VAR}}}"')
self.emit ('return $__awk_rc')
def _awk_helper_func (self, func: FunctionDecl):
"""Generate AWK helper function definition."""
params = ", ".join (p.name for p in func.params)
self.emit (f"function {func.name}({params}) {{")
lines, emit, inc, dec = self._awk_make_emitter ()
for stmt in func.body.statements:
self._awk_stmt (stmt, emit, inc, dec, in_func=True)
for line in lines:
self.emit (f" {self._awk_escape (line)}")
self.emit ("}")
def _awk_stmt (self, stmt, emit, inc, dec, in_func=False):
"""Generate AWK statement."""
if isinstance (stmt, FunctionDecl):
return
if isinstance (stmt, Assignment):
target = self._awk_target (stmt.target)
if isinstance (stmt.value, (ArrayLiteral, DictLiteral)):
var_types = getattr (self, '_awk_var_types', {})
if isinstance (stmt.target, Identifier):
if isinstance (stmt.value, ArrayLiteral):
var_types[stmt.target.name] = "array"
else:
var_types[stmt.target.name] = "dict"
self._awk_var_types = var_types
emit (f"delete {target}")
return
if self._awk_handle_map_filter (stmt, target, emit, inc, dec):
return
value = self._awk_expr (stmt.value)
op = stmt.operator
if op == "=":
emit (f"{target} = {value}")
elif op == "+=":
emit (f"{target} += {value}")
elif op == "-=":
emit (f"{target} -= {value}")
elif op == "*=":
emit (f"{target} *= {value}")
elif op == "/=":
emit (f"{target} /= {value}")
elif op == "%=":
emit (f"{target} %= {value}")
elif op == "..=":
emit (f"{target} = {target} {value}")
else:
emit (f"{target} = {value}")
elif isinstance (stmt, ReturnStmt):
if in_func:
if stmt.value:
emit (f"return {self._awk_expr (stmt.value)}")
else:
emit ("return")
else:
if stmt.value:
emit (f"print {self._awk_expr (stmt.value)}")
emit ("exit")
elif isinstance (stmt, IfStmt):
cond = self._awk_cond (stmt.condition)
emit (f"if ({cond}) {{")
inc ()
for s in stmt.then_branch.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
for elif_cond, elif_block in stmt.elif_branches:
emit (f"}} else if ({self._awk_cond (elif_cond)}) {{")
inc ()
for s in elif_block.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
if stmt.else_branch:
emit ("} else {")
inc ()
if isinstance (stmt.else_branch, Block):
for s in stmt.else_branch.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
else:
self._awk_stmt (stmt.else_branch, emit, inc, dec, in_func)
dec ()
emit ("}")
elif isinstance (stmt, WhileStmt):
emit (f"while ({self._awk_cond (stmt.condition)}) {{")
inc ()
for s in stmt.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
elif isinstance (stmt, ForStmt):
var = stmt.variable
if isinstance (stmt.iterable, CallExpr) and isinstance (stmt.iterable.callee, Identifier):
if stmt.iterable.callee.name == "range":
args = stmt.iterable.arguments
if len (args) == 1:
start, end, step = "0", self._awk_expr (args[0]), "1"
elif len (args) == 2:
start, end, step = self._awk_expr (args[0]), self._awk_expr (args[1]), "1"
else:
start = self._awk_expr (args[0])
end = self._awk_expr (args[1])
step = self._awk_expr (args[2])
emit (f"for ({var} = {start}; {var} < {end}; {var} += {step}) {{")
inc ()
for s in stmt.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
return
arr = self._awk_expr (stmt.iterable)
emit (f"for ({var} in {arr}) {{")
inc ()
for s in stmt.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
elif isinstance (stmt, ForeachStmt):
var = stmt.variables[0]
iterable = stmt.iterable
if isinstance (iterable, CallExpr) and isinstance (iterable.callee, MemberAccess):
if (isinstance (iterable.callee.object, Identifier) and
iterable.callee.member == "split" and
len (iterable.arguments) >= 1):
text_var = iterable.callee.object.name
delim = self._awk_expr (iterable.arguments[0])
emit (f"__n = split({text_var}, __arr, {delim})")
emit ("for (__i = 1; __i <= __n; __i++) {")
inc ()
emit (f"{var} = __arr[__i]")
for s in stmt.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
return
arr = self._awk_expr (iterable)
if len (stmt.variables) > 1:
key_var = stmt.variables[0]
val_var = stmt.variables[1]
emit (f"for ({key_var} in {arr}) {{")
inc ()
emit (f"{val_var} = {arr}[{key_var}]")
for s in stmt.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
else:
emit (f"for ({var} in {arr}) {{")
inc ()
for s in stmt.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
elif isinstance (stmt, ExpressionStmt):
if isinstance (stmt.expression, CallExpr) and isinstance (stmt.expression.callee, Identifier):
func_name = stmt.expression.callee.name
args = stmt.expression.arguments
if func_name == "assert":
cond = self._awk_cond (args[0]) if args else "1"
msg = self._awk_expr (args[1]) if len (args) >= 2 else '"Assertion failed"'
emit (f"if (!({cond})) {{ print {msg} > \"/dev/stderr\"; exit 1 }}")
return
if func_name == "assert_eq":
expected = self._awk_expr (args[0]) if args else '""'
actual = self._awk_expr (args[1]) if len (args) >= 2 else '""'
msg = self._awk_expr (args[2]) if len (args) >= 3 else '"Values not equal"'
emit (f"if ({expected} != {actual}) {{ print {msg} > \"/dev/stderr\"; exit 1 }}")
return
if func_name == "print":
awk_args = [self._awk_expr (a) for a in args]
emit (f"print {', '.join (awk_args)}" if awk_args else "print")
return
if isinstance (stmt.expression, CallExpr) and isinstance (stmt.expression.callee, MemberAccess):
if isinstance (stmt.expression.callee.object, Identifier):
obj_name = stmt.expression.callee.object.name
method = stmt.expression.callee.member
args = stmt.expression.arguments
var_types = getattr (self, '_awk_var_types', {})
var_type = var_types.get (obj_name, "string")
type_name = {"array": "array", "dict": "dict"}.get (var_type, "string")
awk_args = [self._awk_expr (a) for a in args]
awk_code = generate_awk (type_name, method, obj_name, awk_args)
if awk_code:
emit (awk_code)
return
expr = self._awk_expr (stmt.expression)
if expr:
emit (expr)
elif isinstance (stmt, BreakStmt):
emit ("break")
elif isinstance (stmt, ContinueStmt):
emit ("continue")
elif isinstance (stmt, WhenStmt):
val = self._awk_expr (stmt.value)
emit (f"__when_val = {val}")
first = True
for branch in stmt.branches:
if branch.is_else:
emit ("} else {")
else:
conditions = []
for p in branch.patterns:
if isinstance (p, RangePattern):
start = self._awk_expr (p.start)
end = self._awk_expr (p.end)
conditions.append (f"(__when_val >= {start} && __when_val <= {end})")
else:
pval = self._awk_expr (p)
conditions.append (f"__when_val == {pval}")
cond_str = " || ".join (conditions)
if first:
emit (f"if ({cond_str}) {{")
first = False
else:
emit (f"}} else if ({cond_str}) {{")
inc ()
for s in branch.body.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
dec ()
emit ("}")
elif isinstance (stmt, Block):
for s in stmt.statements:
self._awk_stmt (s, emit, inc, dec, in_func)
def _awk_target (self, expr) -> str:
"""Generate AWK assignment target."""
if isinstance (expr, Identifier):
return expr.name
if isinstance (expr, IndexAccess):
obj = self._awk_expr (expr.object)
idx = self._awk_expr (expr.index)
return f"{obj}[{idx}]"
if isinstance (expr, MemberAccess):
if isinstance (expr.object, Identifier) and expr.object.name == "env":
return f'ENVIRON["{expr.member}"]'
obj = self._awk_expr (expr.object)
return f'{obj}["{expr.member}"]'
return self._awk_expr (expr)
def _awk_expr (self, expr) -> str:
"""Generate AWK expression."""
if expr is None:
return ""
if isinstance (expr, IntegerLiteral):
return str (expr.value)
if isinstance (expr, FloatLiteral):
return str (expr.value)
if isinstance (expr, StringLiteral):
if expr.has_interpolation:
return self._awk_interpolate_string (expr.value)
value = expr.value
value = value.replace ('\\', '\\\\')
value = value.replace ('\n', '\\n')
value = value.replace ('\t', '\\t')
value = value.replace ('"', '\\"')
value = value.replace ("'", "\\047")
return f'"{value}"'
if isinstance (expr, BoolLiteral):
return "1" if expr.value else "0"
if isinstance (expr, NilLiteral):
return '""'
if isinstance (expr, Identifier):
return expr.name
if isinstance (expr, ArrayLiteral):
return '""'
if isinstance (expr, DictLiteral):
return '""'
if isinstance (expr, BinaryOp):
left = self._awk_expr (expr.left)
right = self._awk_expr (expr.right)
op = expr.operator
if op == "..":
return f"({left} {right})"
if op in ("==", "!=", "<", ">", "<=", ">=", "&&", "||", "+", "-", "*", "/", "%", "^"):
return f"({left} {op} {right})"
return f"({left} {op} {right})"
if isinstance (expr, UnaryOp):
operand = self._awk_expr (expr.operand)
if expr.operator == "!":
return f"(!{operand})"
if expr.operator == "-":
return f"(-{operand})"
if expr.operator == "++":
return f"(++{operand})"
if expr.operator == "--":
return f"(--{operand})"
return operand
if isinstance (expr, CallExpr):
if isinstance (expr.callee, MemberAccess) and isinstance (expr.callee.object, Identifier):
ns = expr.callee.object.name
method = expr.callee.member
args = expr.arguments
if ns == "math" and method in MATH_METHODS:
math_method = MATH_METHODS[method]
if math_method.awk_builtin:
awk_args = [self._awk_expr(a) for a in args]
return math_method.awk_builtin(awk_args)
if ns == "time" and method in TIME_METHODS:
time_method = TIME_METHODS[method]
if time_method.awk_builtin:
awk_args = [self._awk_expr(a) for a in args]
return time_method.awk_builtin(awk_args)
var_types = getattr (self, '_awk_var_types', {})
var_type = var_types.get (ns, "string")
type_name = {"array": "array", "dict": "dict"}.get(var_type, "string")
awk_args = [self._awk_expr(a) for a in args]
awk_code = generate_awk(type_name, method, ns, awk_args)
if awk_code:
return awk_code
if isinstance (expr.callee, Identifier):
func_name = expr.callee.name
args = [self._awk_expr(a) for a in expr.arguments]
awk_code = get_awk_builtin(func_name, args)
if awk_code:
return awk_code
return f"{func_name}({', '.join(args)})"
return ""
if isinstance (expr, IndexAccess):
obj = self._awk_expr (expr.object)
idx = self._awk_expr (expr.index)
return f"{obj}[{idx}]"
if isinstance (expr, MemberAccess):
if isinstance (expr.object, Identifier) and expr.object.name == "env":
return f'ENVIRON["{expr.member}"]'
obj = self._awk_expr (expr.object)
return f'{obj}["{expr.member}"]'
return ""
def _awk_cond (self, expr) -> str:
"""Generate AWK condition without outer parens."""
result = self._awk_expr (expr)
if result.startswith ('(') and result.endswith (')'):
return result[1:-1]
return result
def _awk_handle_map_filter (self, stmt: Assignment, target: str, emit, inc, dec) -> bool:
"""Handle array.map() and array.filter() calls for assignments."""
if not isinstance (stmt.value, CallExpr):
return False
if not isinstance (stmt.value.callee, MemberAccess):
return False
if not isinstance (stmt.value.callee.object, Identifier):
return False
method = stmt.value.callee.member
arr_name = stmt.value.callee.object.name
args = stmt.value.arguments
if method not in ("map", "filter") or len (args) < 1:
return False
lambda_arg = args[0]
if not isinstance (lambda_arg, Lambda):
return False
var_types = getattr (self, '_awk_var_types', {})
var_types[target] = "array"
self._awk_var_types = var_types
param = lambda_arg.params[0] if lambda_arg.params else "__x"
body_expr = self._awk_lambda_body (lambda_arg.body, param)
emit (f"delete {target}")
if method == "map":
emit (f"__ct_len = length({arr_name})")
emit (f"for (__i = 1; __i <= __ct_len; __i++) {{")
inc ()
emit (f"{param} = {arr_name}[__i]")
emit (f"{target}[__i] = {body_expr}")
dec ()
emit ("}")
elif method == "filter":
emit (f"__ct_len = length({arr_name})")
emit (f"__ct_fidx = 0")
emit (f"for (__i = 1; __i <= __ct_len; __i++) {{")
inc ()
emit (f"{param} = {arr_name}[__i]")
emit (f"if ({body_expr}) {{")
inc ()
emit (f"__ct_fidx++")
emit (f"{target}[__ct_fidx] = {arr_name}[__i]")
dec ()
emit ("}")
dec ()
emit ("}")
return True
def _awk_lambda_body (self, body, param: str) -> str:
"""Generate AWK expression for lambda body."""
if isinstance (body, Block):
for stmt in body.statements:
if isinstance (stmt, ReturnStmt) and stmt.value:
return self._awk_expr (stmt.value)
return '""'
return self._awk_expr (body)
def _awk_interpolate_string (self, value: str) -> str:
"""Convert string with {var} interpolation to AWK concatenation."""
parts = []
i = 0
while i < len (value):
if value[i] == '{' and (i == 0 or value[i - 1] != '\\'):
end = value.find ('}', i)
if end != -1:
if i > 0 and parts == []:
parts.append (f'"{self._awk_escape_str (value[:i])}"')
elif i > 0:
text_before = value[parts[-1][1] if isinstance (parts[-1], tuple) else 0:i]
if text_before and not text_before.startswith ('{'):
pass
var_name = value[i + 1:end]
if '.' in var_name:
obj, member = var_name.split ('.', 1)
if obj == 'env':
parts.append (f'ENVIRON["{member}"]')
else:
parts.append (f'{obj}["{member}"]')
else:
parts.append (var_name)
i = end + 1
continue
i += 1
if not parts:
escaped = self._awk_escape_str (value)
return f'"{escaped}"'
result_parts = []
last_end = 0
i = 0
while i < len (value):
if value[i] == '{' and (i == 0 or value[i - 1] != '\\'):
end = value.find ('}', i)
if end != -1:
if i > last_end:
text = value[last_end:i]
if text:
result_parts.append (f'"{self._awk_escape_str (text)}"')
var_name = value[i + 1:end]
if '.' in var_name:
obj, member = var_name.split ('.', 1)
if obj == 'env':
result_parts.append (f'ENVIRON["{member}"]')
else:
result_parts.append (f'{obj}["{member}"]')
else:
result_parts.append (var_name)
last_end = end + 1
i = end + 1
continue
i += 1
if last_end < len (value):
text = value[last_end:]
if text:
result_parts.append (f'"{self._awk_escape_str (text)}"')
if len (result_parts) == 1:
return result_parts[0]
return '(' + ' '.join (result_parts) + ')'
def _awk_escape_str (self, s: str) -> str:
"""Escape string for AWK double quotes."""
s = s.replace ('\\', '\\\\')
s = s.replace ('\n', '\\n')
s = s.replace ('\t', '\\t')
s = s.replace ('"', '\\"')
s = s.replace ("'", "\\047")
return s
def _awk_emit_validation (self, params, emit):
"""Generate AWK validation code for @validate decorator."""
validate = self._awk_validate
if not validate:
return
validations = {}
for arg_name, arg_val in validate.arguments:
if arg_name and hasattr (arg_val, 'value'):
validations[arg_name] = arg_val.value
for param in params:
rule = validations.get (param.name)
if not rule:
continue
pname = param.name
if "int" in rule:
emit (f'if ({pname} !~ /^-?[0-9]+$/) {{ print "{pname} must be integer" > "/dev/stderr"; exit 1 }}')
for op, val in re.findall (r'(>|<|>=|<=|==|!=)\s*(-?\d+)', rule):
awk_cond = f"{pname} {op} {val}"
emit (f'if (!({awk_cond})) {{ print "{pname} must be {op} {val}" > "/dev/stderr"; exit 1 }}')
bootstrap/class_codegen.py deleted 100644 → 0
View file @ 5f7339cd
from .ast_nodes import (
ClassDecl, ClassField, FunctionDecl, ArrayLiteral, DictLiteral, NilLiteral, NewExpr,
CallExpr, Identifier, Assignment, MemberAccess, ThisExpr, ReturnStmt,
ConstructorDecl, Parameter, Block, ForeachStmt, IfStmt, WhileStmt, ForStmt,
ExpressionStmt, BinaryOp, IndexAccess, TypeAnnotation
)
from .methods import ARRAY_METHODS, DICT_METHODS
ARRAY_ONLY_METHODS = {"push", "pop", "shift", "join", "slice", "map", "filter"}
DICT_ONLY_METHODS = {"has", "del", "keys"}
ARRAY_METHODS_ALL = ARRAY_ONLY_METHODS | {"get", "set", "len"}
DICT_METHODS_ALL = DICT_ONLY_METHODS | {"get", "set", "len"}
STRING_METHODS_ALL = {"upper", "lower", "trim", "len", "contains", "starts", "ends",
"index", "replace", "substr", "split", "charAt", "urlencode"}
ALL_KNOWN_METHODS = ARRAY_METHODS_ALL | DICT_METHODS_ALL | STRING_METHODS_ALL
class ClassMixin:
"""Mixin for class and method generation."""
def _is_method_used(self, cls_name, method_name):
if cls_name in self.used_methods and method_name in self.used_methods[cls_name]:
return True
if "__" in cls_name:
unprefixed = cls_name.split("__", 1)[1]
if unprefixed in self.used_methods and method_name in self.used_methods[unprefixed]:
return True
return False
def _get_field_info(self, field):
if isinstance(field, ClassField):
return field.name, field.type_annotation, field.default
else:
return field[0], None, field[1] if len(field) > 1 else None
def generate_class(self, cls: ClassDecl):
if self.current_namespace:
effective_name = f"{self.current_namespace}__{cls.name}"
parent = cls.parent
if parent:
ns_parent = f"{self.current_namespace}__{parent}"
if ns_parent in self.classes:
parent = ns_parent
cls = ClassDecl(
name=effective_name,
parent=parent,
fields=cls.fields,
constructor=cls.constructor,
methods=cls.methods,
location=cls.location
)
self.current_class = cls.name
self.current_class_fields = set(self._get_field_info(f)[0] for f in cls.fields)
for field in cls.fields:
field_name, type_annotation, default_value = self._get_field_info(field)
if type_annotation:
if type_annotation.name == "array" or type_annotation.is_array:
self.class_field_types[(cls.name, field_name)] = "array"
elif type_annotation.name == "dict":
self.class_field_types[(cls.name, field_name)] = "dict"
elif type_annotation.name in self.classes:
self.class_field_types[(cls.name, field_name)] = "object"
self.class_field_class[(cls.name, field_name)] = type_annotation.name
else:
self.class_field_types[(cls.name, field_name)] = "scalar"
elif isinstance(default_value, ArrayLiteral):
self.class_field_types[(cls.name, field_name)] = "array"
elif isinstance(default_value, DictLiteral):
self.class_field_types[(cls.name, field_name)] = "dict"
elif isinstance(default_value, NilLiteral):
self.class_field_types[(cls.name, field_name)] = "object"
elif isinstance(default_value, NewExpr):
self.class_field_types[(cls.name, field_name)] = "object"
elif isinstance(default_value, CallExpr) and isinstance(default_value.callee, Identifier):
callee_name = default_value.callee.name
if callee_name in self.classes:
self.class_field_types[(cls.name, field_name)] = "object"
else:
self.class_field_types[(cls.name, field_name)] = "scalar"
else:
self.class_field_types[(cls.name, field_name)] = "scalar"
if cls.parent and cls.parent in self.classes:
parent_chain = self._collect_parent_fields(cls.parent)
own_field_names = set(self._get_field_info(f)[0] for f in cls.fields)
for parent_name, parent_fields in parent_chain:
for field in parent_fields:
field_name, _, _ = self._get_field_info(field)
if field_name not in own_field_names:
parent_type = self.class_field_types.get((parent_name, field_name))
if parent_type:
self.class_field_types[(cls.name, field_name)] = parent_type
self.current_class_fields.add(field_name)
for method in cls.methods:
self._check_inlineable_method(cls, method)
self._generate_class_constructor(cls)
self._generate_class_metadata(cls)
if cls.constructor:
self._generate_construct_method(cls)
for method in cls.methods:
if self.used_methods is not None and not self.current_namespace:
if not self._is_method_used(cls.name, method.name):
continue
has_awk = any(dec.name == "awk" for dec in method.decorators)
if has_awk:
self._generate_awk_method(cls, method)
elif method.decorators:
self._generate_decorated_method(cls, method)
else:
self._generate_plain_method(cls, method)
if cls.parent and cls.parent in self.classes:
parent_cls = self.classes[cls.parent]
own_method_names = {m.name for m in cls.methods}
for parent_method in parent_cls.methods:
if parent_method.name not in own_method_names:
if self.used_methods is not None and not self.current_namespace:
if not self._is_method_used(cls.name, parent_method.name):
continue
self._generate_inherited_method(cls, parent_cls, parent_method)
self.current_class = None
self.current_class_fields = set()
def _collect_parent_fields(self, cls_name):
"""Collect all fields from parent chain (top-down order)."""
fields = []
visited = set()
current = cls_name
while current and current in self.classes:
if current in visited:
break
visited.add(current)
parent_cls = self.classes[current]
fields.append((current, parent_cls.fields))
current = parent_cls.parent
fields.reverse()
return fields
def _generate_class_constructor(self, cls: ClassDecl):
"""Generate class factory function."""
self.emit(f"{cls.name} () {{")
with self.indented():
self.emit('local __ct_this_instance="__ct_inst_$RANDOM$RANDOM"')
self.emit('__ct_obj_class["$__ct_this_instance"]="{}"'.format(cls.name))
own_field_names = set(self._get_field_info(f)[0] for f in cls.fields)
if cls.parent and cls.parent in self.classes:
parent_chain = self._collect_parent_fields(cls.parent)
for _parent_name, parent_fields in parent_chain:
for field in parent_fields:
field_name, type_annotation, default_value = self._get_field_info(field)
if field_name in own_field_names:
continue
self._emit_field_init(field_name, type_annotation, default_value)
for field in cls.fields:
field_name, type_annotation, default_value = self._get_field_info(field)
self._emit_field_init(field_name, type_annotation, default_value)
if cls.constructor:
self.emit("# Call constructor")
params_list = " ".join([f'"${{{i + 1}}}"' for i in range(len(cls.constructor.params))])
self.emit(f'__ct_class_{cls.name}_construct "$__ct_this_instance" {params_list}')
self.emit('__ct_last_instance="$__ct_this_instance"')
self.emit("}")
self.emit()
def _emit_field_init(self, field_name, type_annotation, default_value):
"""Emit field initialization in class factory function."""
if isinstance(default_value, ArrayLiteral):
elements = [self.generate_expr(e) for e in default_value.elements]
if elements:
arr_content = " ".join([f'"{e}"' for e in elements])
self.emit(f'declare -ga "${{__ct_this_instance}}_{field_name}=({arr_content})"')
else:
self.emit(f'declare -ga "${{__ct_this_instance}}_{field_name}=()"')
elif isinstance(default_value, DictLiteral):
self.emit(f'eval "declare -gA ${{__ct_this_instance}}_{field_name}=()"')
self.emit(f'__CT_OBJ["$__ct_this_instance.{field_name}"]="${{__ct_this_instance}}_{field_name}"')
elif default_value:
val = self.generate_expr(default_value)
self.emit(f'__CT_OBJ["$__ct_this_instance.{field_name}"]="{val}"')
else:
self.emit(f'__CT_OBJ["$__ct_this_instance.{field_name}"]=""')
def _generate_class_metadata(self, cls: ClassDecl):
field_names = []
field_types = {}
for field in cls.fields:
field_name, type_annotation, _ = self._get_field_info(field)
field_names.append(field_name)
ft = self.class_field_types.get((cls.name, field_name), "scalar")
if ft == "scalar" and type_annotation:
field_types[field_name] = type_annotation.name
elif ft == "scalar":
field_types[field_name] = "string"
elif ft == "object" and type_annotation and type_annotation.name in self.classes:
field_types[field_name] = type_annotation.name
else:
field_types[field_name] = ft
fields_str = " ".join([f'"{f}"' for f in field_names])
self.emit(f'declare -ga __ct_class_meta_{cls.name}_fields=({fields_str})')
types_pairs = " ".join([f'["{f}"]="{t}"' for f, t in field_types.items()])
self.emit(f'declare -gA __ct_class_meta_{cls.name}_types=({types_pairs})')
self.emit()
def _generate_construct_method(self, cls: ClassDecl):
"""Generate construct method."""
self.emit(f"__ct_class_{cls.name}_construct () {{")
with self.indented():
self.emit('local this="$1"')
self.emit('shift')
self.in_class_method = True
old_in_function = self.in_function
old_local_vars = self.local_vars.copy()
old_param_positions = self.current_param_positions.copy()
self.in_function = True
self.local_vars = set()
self.current_param_positions = {}
for i, param in enumerate(cls.constructor.params):
self.current_param_positions[param.name] = i + 1
array_field_params = self._find_array_field_params(cls)
for i, param in enumerate(cls.constructor.params):
if param.name in array_field_params:
continue
if param.default is not None:
default_val = self.generate_expr(param.default)
self.emit(f'local {param.name}="${{{i + 1}:-{default_val}}}"')
else:
self.emit(f'local {param.name}="${{{i + 1}}}"')
self.local_vars.add(param.name)
for stmt in cls.constructor.body.statements:
self.generate_statement(stmt)
self.in_class_method = False
self.in_function = old_in_function
self.local_vars = old_local_vars
self.current_param_positions = old_param_positions
self.emit("}")
self.emit()
def _find_array_field_params(self, cls: ClassDecl) -> set:
"""Find parameters used for array field assignments."""
result = set()
if not cls.constructor:
return result
for stmt in cls.constructor.body.statements:
if isinstance(stmt, Assignment):
if isinstance(stmt.target, MemberAccess) and isinstance(stmt.target.object, ThisExpr):
field = stmt.target.member
field_type = self.class_field_types.get((cls.name, field))
if field_type == "array" and isinstance(stmt.value, Identifier):
result.add(stmt.value.name)
return result
def _generate_plain_method(self, cls: ClassDecl, method: FunctionDecl):
"""Generate a plain class method."""
self.emit(f"__ct_class_{cls.name}_{method.name} () {{")
with self.indented():
self.emit('local this="$1"')
self.emit('shift')
self.in_class_method = True
old_in_function = self.in_function
old_local_vars = self.local_vars.copy()
old_object_vars = self.object_vars.copy()
self.in_function = True
self.local_vars = set()
param_types = self._analyze_param_types(method)
for i, param in enumerate(method.params):
if param.is_variadic:
self.emit(f'local -a {param.name}=("${{@:{i + 1}}}")')
else:
if param.default is not None:
default_val = self.generate_expr(param.default)
self.emit(f'local {param.name}="${{{i + 1}:-{default_val}}}"')
else:
self.emit(f'local {param.name}="${{{i + 1}}}"')
self.local_vars.add(param.name)
if param_types.get(param.name) == "object":
self.object_vars.add(param.name)
for stmt in method.body.statements:
self.generate_statement(stmt)
self.in_class_method = False
self.in_function = old_in_function
self.local_vars = old_local_vars
self.object_vars = old_object_vars
self.emit("}")
self.emit()
def _generate_inherited_method(self, child_cls: ClassDecl, parent_cls: ClassDecl, method: FunctionDecl):
"""Generate proxy method for inherited method."""
self.emit(f"__ct_class_{child_cls.name}_{method.name} () {{")
with self.indented():
self.emit(f'__ct_class_{parent_cls.name}_{method.name} "$@"')
self.emit("}")
self.emit()
def _generate_decorated_method(self, cls: ClassDecl, method: FunctionDecl):
"""Generate a class method with decorators."""
base_name = f"__ct_class_{cls.name}_{method.name}"
original_name = f"{base_name}_orig"
self.emit(f"{original_name} () {{")
with self.indented():
self.emit('local this="$1"')
self.emit('shift')
self.in_class_method = True
old_in_function = self.in_function
old_local_vars = self.local_vars.copy()
old_object_vars = self.object_vars.copy()
self.in_function = True
self.local_vars = set()
param_types = self._analyze_param_types(method)
for i, param in enumerate(method.params):
if param.is_variadic:
self.emit(f'local -a {param.name}=("${{@:{i + 1}}}")')
else:
if param.default is not None:
default_val = self.generate_expr(param.default)
self.emit(f'local {param.name}="${{{i + 1}:-{default_val}}}"')
else:
self.emit(f'local {param.name}="${{{i + 1}}}"')
self.local_vars.add(param.name)
if param_types.get(param.name) == "object":
self.object_vars.add(param.name)
for stmt in method.body.statements:
self.generate_statement(stmt)
self.in_class_method = False
self.in_function = old_in_function
self.local_vars = old_local_vars
self.object_vars = old_object_vars
self.emit("}")
self.emit()
current_name = original_name
for i, decorator in enumerate(reversed(method.decorators)):
wrapper_name = f"{base_name}_d{i}"
self.generate_method_decorator_wrapper(decorator, current_name, wrapper_name, method.params)
current_name = wrapper_name
self.emit(f"{base_name} () {{")
with self.indented():
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(method.params) + 1)])
self.emit(f'{current_name} {params_str}')
self.emit("}")
self.emit()
def _generate_awk_method(self, cls: ClassDecl, method: FunctionDecl):
"""Generate @awk method for class."""
awk_func_name = f"__ct_class_{cls.name}_{method.name}_awk"
method_name = f"__ct_class_{cls.name}_{method.name}"
temp_func = FunctionDecl(
name=awk_func_name,
params=method.params,
body=method.body,
decorators=[],
location=method.location
)
self.generate_awk_function(temp_func)
self.emit(f"# @awk method - 'this' is ignored, calls AWK function")
self.emit(f"{method_name} () {{")
with self.indented():
self.emit('shift # ignore this')
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(method.params))])
self.emit(f'{awk_func_name} {params_str}')
self.emit("}")
self.emit()
def _check_inlineable_method(self, cls: ClassDecl, method: FunctionDecl):
"""Check if method can be inlined."""
if method.params:
return
if len(method.body.statements) != 1:
return
stmt = method.body.statements[0]
if not isinstance(stmt, ReturnStmt) or not stmt.value:
return
value = stmt.value
if isinstance(value, MemberAccess) and isinstance(value.object, ThisExpr):
field = value.member
field_names = [self._get_field_info(f)[0] for f in cls.fields]
if field in field_names:
self.inlineable_methods[(cls.name, method.name)] = f'${{__CT_OBJ["$this.{field}"]}}'
return
if isinstance(value, CallExpr) and isinstance(value.callee, MemberAccess):
if isinstance(value.callee.object, Identifier) and value.callee.object.name == "str":
if value.callee.member == "charAt" and len(value.arguments) == 2:
arg0, arg1 = value.arguments
if isinstance(arg0, MemberAccess) and isinstance(arg0.object, ThisExpr):
if isinstance(arg1, MemberAccess) and isinstance(arg1.object, ThisExpr):
self.inlineable_methods[(cls.name, method.name)] = \
f'${{__CT_OBJ["$this.{arg0.member}"]:${{__CT_OBJ["$this.{arg1.member}"]}}:1}}'
def _analyze_param_types(self, func: FunctionDecl) -> dict:
"""Analyze function body to determine parameter types (array/dict/scalar)."""
param_names = {p.name for p in func.params}
param_types = {}
param_methods = {p.name: set() for p in func.params}
for p in func.params:
if p.type_annotation:
if p.type_annotation.name == "array" or p.type_annotation.is_array:
param_types[p.name] = "array"
elif p.type_annotation.name == "dict":
param_types[p.name] = "dict"
elif p.type_annotation.name in self.classes:
param_types[p.name] = "object"
else:
param_types[p.name] = "scalar"
else:
param_types[p.name] = "scalar"
def analyze_expr(expr):
if isinstance(expr, CallExpr) and isinstance(expr.callee, MemberAccess):
if isinstance(expr.callee.object, Identifier):
var_name = expr.callee.object.name
method = expr.callee.member
if var_name in param_names:
param_methods[var_name].add(method)
for arg in expr.arguments:
analyze_expr(arg)
elif isinstance(expr, BinaryOp):
analyze_expr(expr.left)
analyze_expr(expr.right)
elif isinstance(expr, IndexAccess):
if isinstance(expr.object, Identifier):
var_name = expr.object.name
if var_name in param_names and param_types[var_name] == "scalar":
param_types[var_name] = "array"
analyze_expr(expr.index)
def analyze_stmt(stmt):
if isinstance(stmt, Assignment):
analyze_expr(stmt.value)
if isinstance(stmt.target, IndexAccess):
if isinstance(stmt.target.object, Identifier):
var_name = stmt.target.object.name
if var_name in param_names:
param_types[var_name] = "array"
elif isinstance(stmt, ExpressionStmt):
analyze_expr(stmt.expression)
elif isinstance(stmt, ForeachStmt):
if isinstance(stmt.iterable, Identifier):
var_name = stmt.iterable.name
if var_name in param_names:
param_types[var_name] = "array"
if stmt.body:
for s in stmt.body.statements:
analyze_stmt(s)
elif isinstance(stmt, (IfStmt,)):
analyze_expr(stmt.condition)
if stmt.then_branch:
for s in stmt.then_branch.statements:
analyze_stmt(s)
for _, branch in stmt.elif_branches:
for s in branch.statements:
analyze_stmt(s)
if stmt.else_branch:
for s in stmt.else_branch.statements:
analyze_stmt(s)
elif isinstance(stmt, (WhileStmt, ForStmt)):
if hasattr(stmt, 'condition'):
analyze_expr(stmt.condition)
if stmt.body:
for s in stmt.body.statements:
analyze_stmt(s)
elif isinstance(stmt, ReturnStmt) and stmt.value:
analyze_expr(stmt.value)
if func.body:
for stmt in func.body.statements:
analyze_stmt(stmt)
explicit_typed = {p.name for p in func.params if p.type_annotation}
for param_name, methods in param_methods.items():
if param_name in explicit_typed:
continue
unknown_methods = methods - ALL_KNOWN_METHODS
if unknown_methods:
param_types[param_name] = "object"
continue
if methods & ARRAY_ONLY_METHODS:
param_types[param_name] = "array"
elif methods & DICT_ONLY_METHODS:
param_types[param_name] = "dict"
elif methods & ARRAY_METHODS_ALL and not (methods & DICT_METHODS_ALL - {"get", "set", "len"}):
param_types[param_name] = "array"
return param_types
def generate_function(self, func: FunctionDecl):
test_decorator = None
user_decorators = []
builtin_decorators = []
for dec in func.decorators:
if dec.name == "test":
test_decorator = dec
elif dec.object is not None:
user_decorators.append(dec)
else:
builtin_decorators.append(dec)
if test_decorator:
if not self.test_mode:
self.emit(f"# DCE: skipped @test function {func.name}")
return
description = func.name
if test_decorator.arguments:
arg_name, arg_val = test_decorator.arguments[0]
if hasattr(arg_val, 'value'):
description = arg_val.value
self.test_functions.append((func.name, description))
func = FunctionDecl(
name=func.name,
params=func.params,
body=func.body,
decorators=builtin_decorators,
location=func.location
)
for dec in func.decorators:
if dec.name == "awk":
self.generate_awk_function(func)
self._emit_user_decorators(user_decorators, func.name)
return
if func.decorators:
self.generate_decorated_function(func)
self._emit_user_decorators(user_decorators, func.name)
return
name = func.name
if self.current_namespace and not self.current_class:
name = f"{self.current_namespace}__{func.name}"
elif self.current_class:
name = f"__ct_class_{self.current_class}_{func.name}"
self.emit(f"{name} () {{")
with self.indented():
param_types = self._analyze_param_types(func)
old_param_name_map = getattr(self, 'param_name_map', {})
self.param_name_map = {}
for i, param in enumerate(func.params):
ptype = param_types.get(param.name, "scalar")
if param.is_variadic:
self.emit(f'local -a {param.name}=("${{@:{i + 1}}}")')
elif ptype in ("array", "dict"):
nameref_name = f"__ct_{func.name}_{param.name}"
self.emit(f'local -n {nameref_name}="${{{i + 1}}}"')
self.param_name_map[param.name] = nameref_name
if ptype == "array":
self.array_vars.add(nameref_name)
else:
self.dict_vars.add(nameref_name)
else:
if param.default is not None:
default_val = self.generate_expr(param.default)
self.emit(f'local {param.name}="${{{i + 1}:-{default_val}}}"')
else:
self.emit(f'local {param.name}="${{{i + 1}}}"')
old_deferred = self.deferred_calls
self.deferred_calls = []
old_in_function = self.in_function
self.in_function = True
old_local_vars = self.local_vars
self.local_vars = set()
old_object_vars = self.object_vars.copy()
for param in func.params:
self.local_vars.add(param.name)
if param_types.get(param.name) == "object":
self.object_vars.add(param.name)
for stmt in func.body.statements:
self.generate_statement(stmt)
if self.deferred_calls:
self.emit("# Deferred calls")
for call in reversed(self.deferred_calls):
self.emit(call)
self.deferred_calls = old_deferred
self.in_function = old_in_function
self.local_vars = old_local_vars
self.object_vars = old_object_vars
self.param_name_map = old_param_name_map
self.emit("}")
self.emit()
self._emit_user_decorators(user_decorators, func.name)
def _emit_user_decorators(self, decorators: list, func_name: str):
"""Emit calls for user-defined decorators like @bot.command('start')."""
for dec in decorators:
obj = dec.object
method = dec.name
args = []
for arg_name, arg_val in dec.arguments:
val = self.generate_expr(arg_val)
if not val.startswith('"') and not val.startswith('$'):
val = f'"{val}"'
args.append(val)
args.append(f'"{func_name}"')
args_str = " ".join(args)
obj_class = self.instance_vars.get(obj)
if obj_class:
self.emit(f'__ct_class_{obj_class}_{method} "${obj}" {args_str}')
else:
self.emit(f'{obj}_{method} {args_str}')
def generate_decorated_function(self, func: FunctionDecl) -> str:
original_name = f"__ct_orig_{func.name}"
temp_func = FunctionDecl(
name=original_name,
params=func.params,
body=func.body,
decorators=[],
location=func.location
)
self.generate_function(temp_func)
current_name = original_name
for i, decorator in enumerate(reversed(func.decorators)):
wrapper_name = f"__ct_decorated_{func.name}_{i}"
self.generate_decorator_wrapper(decorator, current_name, wrapper_name, func.params)
current_name = wrapper_name
self.emit(f"{func.name} () {{")
with self.indented():
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(func.params))])
self.emit(f'{current_name} {params_str}')
self.emit("}")
self.emit()
return current_name
bootstrap/codegen.py deleted 100644 → 0
View file @ 5f7339cd
from typing import List, Dict, Optional, Set
from .ast_nodes import Program, ClassDecl, FunctionDecl, Assignment, Identifier, NamespaceDecl, UsingStmt, BusingStmt
from .errors import ErrorCollector
from .stdlib import StdlibMixin
from .awk_codegen import AwkCodegenMixin
from .expr_codegen import ExprMixin
from .stmt_codegen import StmtMixin
from .class_codegen import ClassMixin
from .decorator_codegen import DecoratorMixin
from .dispatch_codegen import DispatchMixin
from .cse_codegen import CseMixin
from .dce import UsageAnalyzer
class CodeGenerator(StdlibMixin, AwkCodegenMixin, ExprMixin, StmtMixin,
ClassMixin, DecoratorMixin, DispatchMixin, CseMixin):
"""
Main code generator class.
Combines mixins:
- StdlibMixin: stdlib function generation
- AwkCodegenMixin: @awk decorator compilation
- ExprMixin: expression generation
- StmtMixin: statement generation
- ClassMixin: class and method generation
- DecoratorMixin: decorator wrapper generation
- DispatchMixin: method dispatch and assignments
- CseMixin: common subexpression elimination
"""
def __init__(self, type_check: bool = True, warn_types: bool = False):
self.output: List[str] = []
self.indent_level = 0
self.errors = ErrorCollector()
self.type_check = type_check
self.warn_types = warn_types
self.current_class: Optional[str] = None
self.current_class_fields: Set[str] = set()
self.in_class_method = False
self.in_function = False
self.classes: Dict[str, ClassDecl] = {}
self.functions: Dict[str, FunctionDecl] = {}
self.inlineable_methods: Dict[tuple, str] = {}
self.test_mode: bool = False
self.test_functions: List[tuple] = []
self.array_vars: Set[str] = set()
self.dict_vars: Set[str] = set()
self.object_vars: Set[str] = set()
self.file_handle_vars: Set[str] = set()
self.process_handle_vars: Set[str] = set()
self.process_handle_map: Dict[str, str] = {}
self.coproc_counter = 0
self.nameref_vars: Set[str] = set() # vars that are namerefs to arrays/dicts
self.callback_vars: Set[str] = set() # vars that hold function names (callbacks)
self.instance_vars: Dict[str, str] = {} # var_name -> class_name
self.class_field_types: Dict[tuple, str] = {}
self.class_field_class: Dict[tuple, str] = {}
self.func_param_types: Dict[tuple, str] = {} # (func_name, param_name) -> "array"/"dict"
self.current_namespace: Optional[str] = None
self.namespaces: Dict[str, dict] = {}
self.busing_names: Dict[str, str] = {}
self.busing_sources: List[str] = []
self.using_direct: Set[str] = set()
self.using_aliases: Dict[str, str] = {}
self.using_selective: Dict[str, Set[str]] = {}
self.local_vars: Set[str] = set()
self.current_param_positions: Dict[str, int] = {} # param_name -> position (1-based)
self.global_vars: Set[str] = {
'L_SRC', 'L_POS', 'L_LEN', 'L_LINE', 'L_COL', 'L_FILE',
'T_TYPES', 'T_VALUES', 'T_LINES', 'T_COUNT',
'P_POS', 'P_STACK',
'A_TYPES', 'A_V1', 'A_V2', 'A_V3', 'A_V4', 'A_V5', 'A_V6', 'A_COUNT',
'G_OUT', 'G_INDENT', 'G_IN_FUNC', 'G_IN_CLASS', 'G_CUR_CLASS',
'G_DEFERRED', 'G_CLASSES', 'G_FUNCS', 'G_ARRAY_VARS', 'G_LOCAL_VARS', 'G_GLOBAL_VARS',
'G_TEMP_CTR', 'G_LAMBDA_CTR',
'G_CSE_KEYS', 'G_CSE_VALS', 'G_CSE_REGEN', 'G_CSE_CALLS',
'G_AWK_LINES', 'G_AWK_INDENT',
'__CT_RET',
}
self.lambda_counter = 0
self.temp_counter = 0
self.deferred_calls: List[str] = []
self.used_classes: Optional[Set[str]] = None
self.used_methods: Optional[Dict[str, Set[str]]] = None
def indent(self) -> str:
return " " * self.indent_level
class _IndentContext:
def __init__(self, gen):
self.gen = gen
def __enter__(self):
self.gen.indent_level += 1
return self
def __exit__(self, *_):
self.gen.indent_level -= 1
def indented(self):
"""Context manager for indented code blocks."""
return self._IndentContext(self)
def emit(self, line: str = ""):
if line:
self.output.append(f"{self.indent()}{line}")
else:
self.output.append("")
def emit_raw(self, line: str):
self.output.append(line)
def emit_var_assign(self, var_name: str, value: str, is_array: bool = False):
"""Emit variable assignment with proper local declaration."""
if '.' in var_name or '[' in var_name:
self.emit(f'{var_name}="{value}"')
return
if self.in_function and var_name not in self.local_vars and var_name not in self.global_vars:
self.local_vars.add(var_name)
if is_array:
self.emit(f'local -a {var_name}=("{value}")')
else:
self.emit(f'local {var_name}="{value}"')
else:
self.emit(f'{var_name}="{value}"')
def new_temp(self) -> str:
self.temp_counter += 1
return f"__ct_tmp_{self.temp_counter}"
def new_lambda_name(self) -> str:
self.lambda_counter += 1
return f"__ct_lambda_{self.lambda_counter}"
def generate(self, program: Program) -> str:
"""Generate code for a single program."""
return self.generate_multi([program])
def generate_multi(self, programs: list, dce: bool = True, test_mode: bool = False) -> str:
"""Generate code for multiple programs (multi-file compilation).
Args:
programs: List of Program AST nodes
dce: Enable Dead Code Elimination (default True)
test_mode: Generate test runner instead of normal execution
"""
self.test_mode = test_mode
self.test_functions = []
self.emit_raw("#!/usr/bin/env bash")
self.emit_raw("# Generated by ContenT compiler")
self.emit_raw("set -euo pipefail")
self.emit()
if dce:
analyzer = UsageAnalyzer()
used_categories = analyzer.analyze(programs, test_mode=test_mode)
if test_mode:
used_categories.add('test')
self.emit_stdlib(used_categories)
self.used_classes = analyzer.get_used_classes()
self.used_methods = analyzer.used_methods
else:
self.emit_stdlib()
self.used_classes = None
self.used_methods = None
for program in programs:
for stmt in program.statements:
if isinstance(stmt, NamespaceDecl):
ns_name = stmt.name
if ns_name not in self.namespaces:
self.namespaces[ns_name] = {"functions": set(), "classes": set()}
for s in stmt.statements:
if isinstance(s, ClassDecl):
prefixed = f"{ns_name}__{s.name}"
self.classes[prefixed] = ClassDecl(
name=prefixed, parent=s.parent,
fields=s.fields, constructor=s.constructor,
methods=s.methods, location=s.location
)
self.namespaces[ns_name]["classes"].add(s.name)
elif isinstance(s, FunctionDecl):
prefixed = f"{ns_name}__{s.name}"
self.functions[prefixed] = s
self.namespaces[ns_name]["functions"].add(s.name)
elif isinstance(stmt, UsingStmt):
if stmt.alias:
self.using_aliases[stmt.alias] = stmt.namespace
elif stmt.names:
if stmt.namespace not in self.using_selective:
self.using_selective[stmt.namespace] = set()
self.using_selective[stmt.namespace].update(stmt.names)
else:
self.using_direct.add(stmt.namespace)
elif isinstance(stmt, BusingStmt):
self.busing_sources.append(stmt.path)
if stmt.name:
self.busing_names[stmt.name] = stmt.path
elif isinstance(stmt, ClassDecl):
self.classes[stmt.name] = stmt
elif isinstance(stmt, FunctionDecl):
self.functions[stmt.name] = stmt
elif isinstance(stmt, Assignment):
if isinstance(stmt.target, Identifier):
self.global_vars.add(stmt.target.name)
for cls_name, cls_decl in list(self.classes.items()):
if cls_decl.parent and cls_decl.parent not in self.classes:
for ns in self.namespaces:
ns_parent = f"{ns}__{cls_decl.parent}"
if ns_parent in self.classes:
self.classes[cls_name] = ClassDecl(
name=cls_decl.name, parent=ns_parent,
fields=cls_decl.fields, constructor=cls_decl.constructor,
methods=cls_decl.methods, location=cls_decl.location
)
break
if self.busing_sources:
for path in self.busing_sources:
self.emit(f'source "{path}"')
self.emit()
for program in programs:
for stmt in program.statements:
self.generate_statement(stmt)
if test_mode and self.test_functions:
self._generate_test_runner()
return "\n".join(self.output)
def _resolve_name(self, name: str) -> str:
if name in self.functions or name in self.classes:
return name
if self.current_namespace:
prefixed = f"{self.current_namespace}__{name}"
if prefixed in self.functions or prefixed in self.classes:
return prefixed
for ns in self.using_direct:
prefixed = f"{ns}__{name}"
if prefixed in self.functions or prefixed in self.classes:
return prefixed
for ns, names in self.using_selective.items():
if name in names:
prefixed = f"{ns}__{name}"
if prefixed in self.functions or prefixed in self.classes:
return prefixed
return name
def _resolve_qualified(self, ns_name: str, member: str) -> str:
if ns_name in self.namespaces:
return f"{ns_name}__{member}"
if ns_name in self.using_aliases:
real_ns = self.using_aliases[ns_name]
return f"{real_ns}__{member}"
return None
def _generate_test_runner(self):
"""Generate test runner that executes all @test functions."""
self.emit()
self.emit("# === Test Runner ===")
self.emit(f'echo "Running {len(self.test_functions)} tests..."')
self.emit('echo')
self.emit('__ct_run_tests () {')
with self.indented():
self.emit('local __test_failed=0')
for func_name, description in self.test_functions:
escaped_desc = description.replace('"', '\\"')
self.emit(f'__ct_test_start "{escaped_desc}"')
self.emit('local __prev_failed=$__ct_test_failed')
self.emit(f'if {func_name}; then')
with self.indented():
self.emit('__ct_test_pass')
self.emit('else')
with self.indented():
self.emit('if [[ $__ct_test_failed -eq $__prev_failed ]]; then')
with self.indented():
self.emit('__ct_test_fail ""')
self.emit('fi')
self.emit('__test_failed=1')
self.emit('fi')
self.emit()
self.emit('__ct_test_summary')
self.emit('}')
self.emit()
self.emit('__ct_run_tests')
bootstrap/constants.py deleted 100644 → 0
View file @ 5f7339cd
"""Constants for bash code generation."""
RET_VAR = "__CT_RET"
RET_ARR = "__CT_RET_ARR"
TMP_PREFIX = "__ct_tmp_"
CLASS_FUNC_PREFIX = "__ct_class_"
LAMBDA_PREFIX = "__ct_lambda_"
OBJ_STORE = "__CT_OBJ"
THIS_INSTANCE = "__ct_this_instance"
ARR_FUNC_PREFIX = "__ct_arr_"
DICT_FUNC_PREFIX = "__ct_dict_"
STR_FUNC_PREFIX = "__ct_str_"
FH_FUNC_PREFIX = "__ct_fh_"
HTTP_FUNC_PREFIX = "__ct_http_"
FS_FUNC_PREFIX = "__ct_fs_"
JSON_FUNC_PREFIX = "__ct_json_"
REGEX_FUNC_PREFIX = "__ct_regex_"
MATH_FUNC_PREFIX = "__ct_math_"
COPROC_PREFIX = "__ct_cp"
bootstrap/cse_codegen.py deleted 100644 → 0
View file @ 5f7339cd
from typing import Dict, Any
from .ast_nodes import (
Expression, CallExpr, MemberAccess, ThisExpr, Identifier,
BinaryOp, UnaryOp, BoolLiteral
)
from .constants import RET_VAR
class NodeIdMap:
"""Mapping from AST nodes to values using id() with reference retention."""
def __init__(self):
self._map: Dict[int, Any] = {}
self._refs = []
def set(self, node, value):
self._refs.append(node)
self._map[id(node)] = value
def get(self, node, default=None):
return self._map.get(id(node), default)
def __contains__(self, node):
return id(node) in self._map
def __getitem__(self, node):
return self._map[id(node)]
class CseMixin:
"""Mixin for CSE optimization."""
def collect_method_calls(self, expr: Expression, calls: list):
"""Collect this.method() calls from an expression."""
if isinstance(expr, CallExpr):
if isinstance(expr.callee, MemberAccess) and isinstance(expr.callee.object, ThisExpr):
calls.append(expr)
for arg in expr.arguments:
self.collect_method_calls(arg, calls)
elif isinstance(expr, BinaryOp):
self.collect_method_calls(expr.left, calls)
self.collect_method_calls(expr.right, calls)
elif isinstance(expr, UnaryOp):
self.collect_method_calls(expr.operand, calls)
elif isinstance(expr, MemberAccess):
self.collect_method_calls(expr.object, calls)
def collect_all_calls(self, expr: Expression, calls: list):
"""Collect ALL function calls from an expression."""
if isinstance(expr, CallExpr):
calls.append(expr)
for arg in expr.arguments:
self.collect_all_calls(arg, calls)
elif isinstance(expr, BinaryOp):
self.collect_all_calls(expr.left, calls)
self.collect_all_calls(expr.right, calls)
elif isinstance(expr, UnaryOp):
self.collect_all_calls(expr.operand, calls)
elif isinstance(expr, MemberAccess):
self.collect_all_calls(expr.object, calls)
def precompute_condition_calls(self, condition: Expression) -> tuple:
"""Pre-compute method calls in condition."""
calls = []
self.collect_method_calls(condition, calls)
seen = {}
mapping = NodeIdMap()
regen_code = []
for call in calls:
method = call.callee.member
args = [self.generate_expr(arg) for arg in call.arguments]
args_str = " ".join([f'"{a}"' for a in args])
key = f"this.{method}({args_str})"
if key not in seen:
temp = self.new_temp()
call_line = f'__ct_class_{self.current_class}_{method} "$this" {args_str} >/dev/null'
assign_line = f'{temp}="${{{RET_VAR}}}"'
self.emit(call_line)
self.emit(assign_line)
seen[key] = temp
regen_code.append((call_line, assign_line))
mapping.set(call, seen[key])
return mapping, regen_code
def precompute_all_calls(self, condition: Expression) -> tuple:
"""Pre-compute all function calls in condition."""
calls = []
self.collect_all_calls(condition, calls)
seen = {}
mapping = NodeIdMap()
regen_code = []
for call in calls:
if isinstance(call.callee, MemberAccess):
if isinstance(call.callee.object, ThisExpr):
method = call.callee.member
args = [self.generate_expr(arg) for arg in call.arguments]
args_str = " ".join([f'"{a}"' for a in args])
key = f"this.{method}({args_str})"
if key not in seen:
temp = self.new_temp()
call_line = f'__ct_class_{self.current_class}_{method} "$this" {args_str} >/dev/null'
assign_line = f'{temp}="${{{RET_VAR}}}"'
self.emit(call_line)
self.emit(assign_line)
seen[key] = temp
regen_code.append((call_line, assign_line))
mapping.set(call, seen[key])
elif isinstance(call.callee.object, Identifier):
obj_name = call.callee.object.name
method = call.callee.member
args = [self.generate_expr(arg) for arg in call.arguments]
args_str = " ".join([f'"{a}"' for a in args])
key = f"{obj_name}.{method}({args_str})"
if key not in seen:
temp = self.new_temp()
call_expr = self.generate_expr(call)
if call_expr.startswith('$'):
call_line = f'{temp}="{call_expr}"'
else:
call_line = f'{temp}="$({call_expr})"'
self.emit(call_line)
seen[key] = temp
regen_code.append((call_line, ""))
mapping.set(call, seen[key])
elif isinstance(call.callee, Identifier):
func_name = call.callee.name
if func_name in ('is_empty', 'is_number'):
continue
args = [self.generate_expr(arg) for arg in call.arguments]
args_str = " ".join([f'"{a}"' for a in args])
key = f"{func_name}({args_str})"
if key not in seen:
temp = self.new_temp()
resolved_call = self.generate_call_statement(call)
call_line = f'{resolved_call} >/dev/null'
assign_line = f'{temp}="${{{RET_VAR}}}"'
self.emit(call_line)
self.emit(assign_line)
seen[key] = temp
regen_code.append((call_line, assign_line))
mapping.set(call, seen[key])
return mapping, regen_code
def generate_condition_with_precompute(self, expr: Expression, mapping: dict) -> str:
"""Generate condition using pre-computed values."""
if isinstance(expr, BinaryOp):
left = self.generate_expr_with_precompute(expr.left, mapping)
right = self.generate_expr_with_precompute(expr.right, mapping)
op = expr.operator
if op == "==":
return f'[[ "{left}" == "{right}" ]]'
elif op == "!=":
return f'[[ "{left}" != "{right}" ]]'
elif op == "<":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ "{left}" < "{right}" ]]'
return f'[[ {left} -lt {right} ]]'
elif op == ">":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ "{left}" > "{right}" ]]'
return f'[[ {left} -gt {right} ]]'
elif op == "<=":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ ! "{left}" > "{right}" ]]'
return f'[[ {left} -le {right} ]]'
elif op == ">=":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ ! "{left}" < "{right}" ]]'
return f'[[ {left} -ge {right} ]]'
elif op == "&&":
l = self.generate_condition_with_precompute(expr.left, mapping)
r = self.generate_condition_with_precompute(expr.right, mapping)
return f'{{ {l} && {r}; }}'
elif op == "||":
l = self.generate_condition_with_precompute(expr.left, mapping)
r = self.generate_condition_with_precompute(expr.right, mapping)
return f'{{ {l} || {r}; }}'
if isinstance(expr, UnaryOp) and expr.operator == "!":
inner = self.generate_condition_with_precompute(expr.operand, mapping)
return f'! {inner}'
if isinstance(expr, Identifier):
return f'[[ "${expr.name}" == "true" ]]'
if isinstance(expr, BoolLiteral):
return "true" if expr.value else "false"
if isinstance(expr, CallExpr) and expr in mapping:
return f'[[ "${{{mapping[expr]}}}" == "true" ]]'
return self.generate_condition(expr)
def generate_expr_with_precompute(self, expr: Expression, mapping: NodeIdMap) -> str:
"""Generate expression using pre-computed values."""
if isinstance(expr, CallExpr) and expr in mapping:
return f'${mapping[expr]}'
if isinstance(expr, MemberAccess):
if isinstance(expr.object, CallExpr) and expr.object in mapping:
temp = mapping[expr.object]
return f'${{__CT_OBJ["${temp}.{expr.member}"]}}'
return self.generate_expr(expr)
return self.generate_expr(expr)
bootstrap/dce.py deleted 100644 → 0
View file @ 5f7339cd
"""Dead Code Elimination."""
import re
from .ast_nodes import (
ClassDecl, NewExpr, CallExpr, Identifier, FunctionDecl, Assignment,
ExpressionStmt, IfStmt, ForStmt, ForeachStmt, WhileStmt, WhenStmt,
WhenBranch, TryStmt, ThrowStmt, DeferStmt, AwaitStmt, OnSignalStmt,
AsyncExpr, ReturnStmt, ArrayLiteral,
DictLiteral, IndexAccess, Lambda, MemberAccess, ThisExpr, Block,
BinaryOp, UnaryOp, WithStmt, Program, StringLiteral,
NamespaceDecl, BusingStmt
)
class UsageAnalyzer:
CATEGORIES = {
'core', 'object', 'http', 'fs', 'json', 'logger', 'string',
'array', 'dict', 'regex', 'math', 'time', 'awk', 'exception',
'args', 'misc', 'async',
}
ARRAY_RETURNING_METHODS = {'keys', 'split', 'slice'}
DICT_RETURNING_METHODS = set()
def __init__(self):
self.used: set = set()
self.has_classes = False
self.has_awk = False
self.test_mode = False
self.defined_classes: dict = {}
self.defined_functions: dict = {}
self.used_classes: set = set()
self.used_methods: dict = {}
self.class_fields: dict = {}
self.variable_types: dict = {}
self.array_variables: set = set()
self.dict_variables: set = set()
self.current_class_name: str = None
self.current_method_name: str = None
self.method_calls: dict = {}
self.func_param_types: dict = {}
self.namespaces: dict = {}
self.busing_names: set = set()
def analyze(self, programs: list, test_mode: bool = False) -> set:
self.used = {'core'}
self.test_mode = test_mode
for program in programs:
for stmt in program.statements:
if isinstance(stmt, ClassDecl):
self.defined_classes[stmt.name] = stmt
self._collect_class_fields(stmt)
elif isinstance(stmt, FunctionDecl):
self.defined_functions[stmt.name] = stmt
elif isinstance(stmt, NamespaceDecl):
ns_name = stmt.name
if ns_name not in self.namespaces:
self.namespaces[ns_name] = {"classes": set(), "functions": set()}
for s in stmt.statements:
if isinstance(s, ClassDecl):
prefixed = f"{ns_name}__{s.name}"
self.defined_classes[prefixed] = s
self._collect_class_fields(s)
self.namespaces[ns_name]["classes"].add(s.name)
elif isinstance(s, FunctionDecl):
prefixed = f"{ns_name}__{s.name}"
self.defined_functions[prefixed] = s
self.namespaces[ns_name]["functions"].add(s.name)
elif isinstance(stmt, BusingStmt):
if stmt.name:
self.busing_names.add(stmt.name)
for program in programs:
for stmt in program.statements:
self._analyze_stmt(stmt)
self._resolve_transitive_classes()
self._resolve_transitive_methods()
if self.has_classes:
self.used.add('object')
if self.has_awk:
self.used.add('awk')
return self.used
def _collect_class_fields(self, cls: ClassDecl):
from .ast_nodes import ClassField
self.class_fields[cls.name] = {}
for field in cls.fields:
if isinstance(field, ClassField):
field_name = field.name
default_value = field.default
type_annotation = field.type_annotation
else:
field_name, default_value = field
type_annotation = None
field_class = None
if type_annotation and type_annotation.name in self.defined_classes:
field_class = type_annotation.name
elif default_value:
if isinstance(default_value, NewExpr):
field_class = default_value.class_name
elif isinstance(default_value, CallExpr) and isinstance(default_value.callee, Identifier):
field_class = default_value.callee.name
self.class_fields[cls.name][field_name] = field_class
def _resolve_transitive_classes(self):
changed = True
while changed:
changed = False
for cls_name in list(self.used_classes):
if cls_name in self.defined_classes:
cls_decl = self.defined_classes[cls_name]
if cls_decl.parent and cls_decl.parent not in self.used_classes:
if cls_decl.parent in self.defined_classes:
self.used_classes.add(cls_decl.parent)
changed = True
if cls_name in self.class_fields:
for field_name, field_class in self.class_fields[cls_name].items():
if field_class and field_class not in self.used_classes:
if field_class in self.defined_classes:
self.used_classes.add(field_class)
changed = True
def _resolve_transitive_methods(self):
changed = True
while changed:
changed = False
for cls_name, methods in list(self.used_methods.items()):
for method_name in list(methods):
key = (cls_name, method_name)
if key in self.method_calls:
for called_cls, called_method in self.method_calls[key]:
if called_cls not in self.used_methods:
self.used_methods[called_cls] = set()
if called_method not in self.used_methods[called_cls]:
self.used_methods[called_cls].add(called_method)
changed = True
for cls_name in list(self.used_classes):
if cls_name in self.defined_classes:
cls_decl = self.defined_classes[cls_name]
if cls_decl.parent and cls_decl.parent in self.defined_classes:
parent_cls = self.defined_classes[cls_decl.parent]
if cls_name in self.used_methods:
child_methods = {m.name for m in cls_decl.methods}
for method in list(self.used_methods[cls_name]):
if method not in child_methods:
if cls_decl.parent not in self.used_methods:
self.used_methods[cls_decl.parent] = set()
if method not in self.used_methods[cls_decl.parent]:
self.used_methods[cls_decl.parent].add(method)
changed = True
for cls_name in list(self.used_classes):
if cls_name in self.defined_classes:
cls_decl = self.defined_classes[cls_name]
if cls_decl.parent and cls_decl.parent in self.used_methods:
child_methods = {m.name for m in cls_decl.methods}
for method in list(self.used_methods[cls_decl.parent]):
if method not in child_methods:
if cls_name not in self.used_methods:
self.used_methods[cls_name] = set()
if method not in self.used_methods[cls_name]:
self.used_methods[cls_name].add(method)
changed = True
def get_used_classes(self) -> set:
return self.used_classes
def get_used_methods(self, class_name: str) -> set:
return self.used_methods.get(class_name, set())
def _analyze_stmt(self, stmt):
if isinstance(stmt, NamespaceDecl):
for s in stmt.statements:
self._analyze_stmt(s)
return
if isinstance(stmt, ClassDecl):
self.has_classes = True
self.current_class_name = stmt.name
for method in stmt.methods:
self.current_method_name = method.name
if method.decorators:
for dec in method.decorators:
if dec.name == 'awk':
self.has_awk = True
self._analyze_body(method.body)
self.current_method_name = None
if stmt.constructor:
self._analyze_body(stmt.constructor.body)
self.current_class_name = None
elif isinstance(stmt, FunctionDecl):
is_test = any(dec.name == 'test' for dec in stmt.decorators)
if is_test and not self.test_mode:
return
if stmt.decorators:
for dec in stmt.decorators:
if dec.name == 'awk':
self.has_awk = True
if dec.object:
obj_class = self.variable_types.get(dec.object)
if obj_class:
if obj_class not in self.used_methods:
self.used_methods[obj_class] = set()
self.used_methods[obj_class].add(dec.name)
old_func_name = getattr(self, 'current_func_name', None)
self.current_func_name = stmt.name
self._analyze_body(stmt.body)
self.current_func_name = old_func_name
elif isinstance(stmt, Assignment):
self._analyze_expr(stmt.value)
if isinstance(stmt.target, Identifier):
var_name = stmt.target.name
if isinstance(stmt.value, NewExpr):
self.variable_types[var_name] = stmt.value.class_name
elif isinstance(stmt.value, CallExpr):
if isinstance(stmt.value.callee, Identifier):
callee_name = stmt.value.callee.name
if callee_name in self.defined_classes:
self.variable_types[var_name] = callee_name
elif isinstance(stmt.value.callee, MemberAccess):
method = stmt.value.callee.member
if method in self.ARRAY_RETURNING_METHODS:
self.array_variables.add(var_name)
elif method in self.DICT_RETURNING_METHODS:
self.dict_variables.add(var_name)
elif isinstance(stmt.value, ArrayLiteral):
self.array_variables.add(var_name)
elif isinstance(stmt.value, DictLiteral):
self.dict_variables.add(var_name)
elif isinstance(stmt, ExpressionStmt):
self._analyze_expr(stmt.expression)
elif isinstance(stmt, IfStmt):
self._analyze_expr(stmt.condition)
self._analyze_body(stmt.then_branch)
if stmt.else_branch:
self._analyze_body(stmt.else_branch)
elif isinstance(stmt, ForStmt):
self._analyze_expr(stmt.iterable)
self._analyze_body(stmt.body)
elif isinstance(stmt, ForeachStmt):
self._analyze_expr(stmt.iterable)
self._analyze_body(stmt.body)
elif isinstance(stmt, WhileStmt):
self._analyze_expr(stmt.condition)
self._analyze_body(stmt.body)
elif isinstance(stmt, WhenStmt):
if stmt.value:
self._analyze_expr(stmt.value)
for branch in stmt.branches:
for pattern in branch.patterns:
self._analyze_expr(pattern)
self._analyze_body(branch.body)
elif isinstance(stmt, TryStmt):
self.used.add('exception')
self._analyze_body(stmt.try_block)
for except_clause in stmt.except_clauses:
if len(except_clause) >= 2:
self._analyze_body(except_clause[1])
if stmt.finally_block:
self._analyze_body(stmt.finally_block)
elif isinstance(stmt, ThrowStmt):
self.used.add('exception')
self._analyze_expr(stmt.expression)
elif isinstance(stmt, DeferStmt):
self.used.add('exception')
self._analyze_expr(stmt.expression)
elif isinstance(stmt, AwaitStmt):
self.used.add('async')
self._analyze_expr(stmt.expression)
elif isinstance(stmt, OnSignalStmt):
self.used.add('async')
self._analyze_body(stmt.body)
elif isinstance(stmt, ReturnStmt):
if stmt.value:
self._analyze_expr(stmt.value)
def _analyze_body(self, body):
if body is None:
return
if hasattr(body, 'statements'):
stmts = body.statements
elif isinstance(body, list):
stmts = body
else:
return
for stmt in stmts:
self._analyze_stmt(stmt)
def _analyze_expr(self, expr):
if expr is None:
return
if isinstance(expr, CallExpr):
self._analyze_call(expr)
for arg in expr.arguments:
self._analyze_expr(arg)
elif isinstance(expr, MemberAccess):
self._analyze_member_access(expr)
self._analyze_expr(expr.object)
elif isinstance(expr, BinaryOp):
self._analyze_expr(expr.left)
self._analyze_expr(expr.right)
elif isinstance(expr, UnaryOp):
self._analyze_expr(expr.operand)
elif isinstance(expr, ArrayLiteral):
self.used.add('array')
for elem in expr.elements:
self._analyze_expr(elem)
elif isinstance(expr, DictLiteral):
self.used.add('dict')
for k, v in expr.pairs:
self._analyze_expr(k)
self._analyze_expr(v)
elif isinstance(expr, IndexAccess):
self._analyze_expr(expr.object)
self._analyze_expr(expr.index)
elif isinstance(expr, Lambda):
self._analyze_body(expr.body)
elif isinstance(expr, AsyncExpr):
self.used.add('async')
self._analyze_expr(expr.expression)
elif isinstance(expr, NewExpr):
self.has_classes = True
self.used_classes.add(expr.class_name)
for arg in expr.arguments:
self._analyze_expr(arg)
elif isinstance(expr, Identifier):
pass
elif isinstance(expr, StringLiteral):
if getattr(expr, 'has_interpolation', False):
self._analyze_string_interpolation(expr.value)
def _analyze_string_interpolation(self, value: str):
"""Analyze method calls in string interpolation like {var.method()}."""
pattern = r'\{(\w+)\.(\w+)\s*\([^)]*\)\}'
for match in re.finditer(pattern, value):
var_name = match.group(1)
method = match.group(2)
if var_name in self.variable_types:
obj_class = self.variable_types[var_name]
if obj_class not in self.used_methods:
self.used_methods[obj_class] = set()
self.used_methods[obj_class].add(method)
elif hasattr(self, 'current_func_name') and self.current_func_name:
key = (self.current_func_name, var_name)
if key in self.func_param_types:
for obj_class in self.func_param_types[key]:
if obj_class not in self.used_methods:
self.used_methods[obj_class] = set()
self.used_methods[obj_class].add(method)
else:
for cls_name, cls_decl in self.defined_classes.items():
for m in cls_decl.methods:
if m.name == method:
if cls_name not in self.used_methods:
self.used_methods[cls_name] = set()
self.used_methods[cls_name].add(method)
def _analyze_call(self, expr: CallExpr):
callee = expr.callee
if isinstance(callee, Identifier):
if callee.name in self.defined_classes:
self.has_classes = True
self.used_classes.add(callee.name)
elif callee.name == 'len':
self.used.add('string')
self.used.add('array')
elif callee.name in ('random', 'random_range'):
self.used.add('misc')
elif callee.name in self.defined_functions:
self._analyze_function_call_with_types(callee.name, expr.arguments)
if isinstance(callee, MemberAccess):
if isinstance(callee.object, ThisExpr):
method = callee.member
if self.current_class_name:
if self.current_class_name not in self.used_methods:
self.used_methods[self.current_class_name] = set()
self.used_methods[self.current_class_name].add(method)
if self.current_method_name:
caller = (self.current_class_name, self.current_method_name)
callee_key = (self.current_class_name, method)
if caller not in self.method_calls:
self.method_calls[caller] = set()
self.method_calls[caller].add(callee_key)
elif isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, ThisExpr):
field_name = callee.object.member
method = callee.member
found_field_class = False
for cls_name, fields in self.class_fields.items():
if field_name in fields and fields[field_name]:
field_class = fields[field_name]
if field_class not in self.used_methods:
self.used_methods[field_class] = set()
self.used_methods[field_class].add(method)
found_field_class = True
if not found_field_class:
self._check_method(method)
elif isinstance(callee.object, CallExpr):
self._analyze_call(callee.object)
self._check_method(callee.member)
elif isinstance(callee.object, Identifier):
ns = callee.object.name
method = callee.member
if ns in self.namespaces:
prefixed = f"{ns}__{method}"
if prefixed in self.defined_classes:
self.has_classes = True
self.used_classes.add(prefixed)
return
if ns in self.busing_names:
return
if ns in self.variable_types:
obj_class = self.variable_types[ns]
if obj_class not in self.used_methods:
self.used_methods[obj_class] = set()
self.used_methods[obj_class].add(method)
elif ns in self.array_variables:
self.used.add('array')
elif ns in self.dict_variables:
self.used.add('dict')
elif ns == 'http':
self.used.add('http')
elif ns == 'fs':
self.used.add('fs')
elif ns == 'json':
self.used.add('json')
if method in ('unmarshal',) and len(expr.arguments) >= 2:
arg2 = expr.arguments[1]
if isinstance(arg2, Identifier) and arg2.name in self.defined_classes:
self.has_classes = True
self.used_classes.add(arg2.name)
self.used.add('object')
elif ns == 'logger':
self.used.add('logger')
elif ns == 'regex':
self.used.add('regex')
elif ns == 'math':
self.used.add('math')
elif ns == 'time':
self.used.add('time')
elif ns == 'args':
self.used.add('args')
elif ns == 'reflect':
self.used.add('reflect')
self.used.add('object')
if method == 'create' and len(expr.arguments) >= 1:
arg1 = expr.arguments[0]
cls_ref = None
if isinstance(arg1, Identifier) and arg1.name in self.defined_classes:
cls_ref = arg1.name
elif isinstance(arg1, StringLiteral) and arg1.value in self.defined_classes:
cls_ref = arg1.value
if cls_ref:
self.has_classes = True
self.used_classes.add(cls_ref)
elif ns == 'shell':
pass
elif hasattr(self, 'current_func_name') and self.current_func_name:
key = (self.current_func_name, ns)
if key in self.func_param_types:
for obj_class in self.func_param_types[key]:
if obj_class not in self.used_methods:
self.used_methods[obj_class] = set()
self.used_methods[obj_class].add(method)
else:
self._check_method(method)
for cls_name, cls_decl in self.defined_classes.items():
for m in cls_decl.methods:
if m.name == method:
if cls_name not in self.used_methods:
self.used_methods[cls_name] = set()
self.used_methods[cls_name].add(method)
else:
found_in_class = False
for cls_name, cls_decl in self.defined_classes.items():
for m in cls_decl.methods:
if m.name == method:
if cls_name not in self.used_methods:
self.used_methods[cls_name] = set()
self.used_methods[cls_name].add(method)
found_in_class = True
if not found_in_class:
self._check_method(method)
def _analyze_function_call_with_types(self, func_name: str, arguments: list):
"""Analyze function call and propagate object types to parameters."""
func_decl = self.defined_functions.get(func_name)
if not func_decl:
return
new_types_added = False
for i, arg in enumerate(arguments):
if i >= len(func_decl.params):
break
param_name = func_decl.params[i].name
arg_type = None
if isinstance(arg, Identifier):
arg_type = self.variable_types.get(arg.name)
if not arg_type and hasattr(self, 'current_func_name') and self.current_func_name:
key = (self.current_func_name, arg.name)
if key in self.func_param_types:
for t in self.func_param_types[key]:
arg_type = t
break
elif isinstance(arg, NewExpr):
arg_type = arg.class_name
elif isinstance(arg, CallExpr) and isinstance(arg.callee, Identifier):
if arg.callee.name in self.defined_classes:
arg_type = arg.callee.name
if arg_type and arg_type in self.defined_classes:
key = (func_name, param_name)
if key not in self.func_param_types:
self.func_param_types[key] = set()
if arg_type not in self.func_param_types[key]:
self.func_param_types[key].add(arg_type)
new_types_added = True
if new_types_added:
old_func = getattr(self, 'current_func_name', None)
self.current_func_name = func_name
self._analyze_body(func_decl.body)
self.current_func_name = old_func
def _analyze_member_access(self, expr: MemberAccess):
if isinstance(expr.object, Identifier):
pass
def _check_method(self, method: str):
string_methods = {'upper', 'lower', 'trim', 'len', 'contains', 'starts',
'ends', 'index', 'replace', 'substr', 'split', 'charAt', 'urlencode'}
array_methods = {'push', 'pop', 'shift', 'join', 'get', 'set', 'slice', 'len'}
dict_methods = {'get', 'set', 'has', 'del', 'keys'}
if method in string_methods:
self.used.add('string')
if method in array_methods:
self.used.add('array')
if method in dict_methods:
self.used.add('dict')
bootstrap/decorator_codegen.py deleted 100644 → 0
View file @ 5f7339cd
import re
from typing import List
from .ast_nodes import Decorator, Parameter
from .constants import RET_VAR
class DecoratorMixin:
"""Mixin for decorator wrapper generation."""
def generate_decorator_wrapper(self, decorator: Decorator, wrapped_name: str,
wrapper_name: str, params: List[Parameter]):
"""Generate decorator wrapper for standalone function."""
if decorator.name == "retry":
self._generate_retry_wrapper(decorator, wrapped_name, wrapper_name, params, is_method=False)
elif decorator.name == "log":
self._generate_log_wrapper(wrapped_name, wrapper_name, params, is_method=False)
elif decorator.name == "cache":
self._generate_cache_wrapper(decorator, wrapped_name, wrapper_name, params, is_method=False)
elif decorator.name == "validate":
self._generate_validate_wrapper(decorator, wrapped_name, wrapper_name, params, is_method=False)
else:
self._generate_passthrough_wrapper(wrapped_name, wrapper_name, params, is_method=False)
def generate_method_decorator_wrapper(self, decorator: Decorator, wrapped_name: str,
wrapper_name: str, params: List[Parameter]):
"""Generate decorator wrapper for class method."""
if decorator.name == "retry":
self._generate_retry_wrapper(decorator, wrapped_name, wrapper_name, params, is_method=True)
elif decorator.name == "log":
self._generate_log_wrapper(wrapped_name, wrapper_name, params, is_method=True)
elif decorator.name == "cache":
self._generate_cache_wrapper(decorator, wrapped_name, wrapper_name, params, is_method=True)
elif decorator.name == "validate":
self._generate_validate_wrapper(decorator, wrapped_name, wrapper_name, params, is_method=True)
else:
self._generate_passthrough_wrapper(wrapped_name, wrapper_name, params, is_method=True)
self.current_class = None
self.current_class_fields = set()
def _generate_retry_wrapper(self, decorator: Decorator, wrapped_name: str,
wrapper_name: str, params: List[Parameter], is_method: bool):
attempts = 3
delay = 1
for arg_name, arg_val in decorator.arguments:
if arg_name == "attempts":
attempts = self.generate_expr(arg_val)
elif arg_name == "delay":
delay = self.generate_expr(arg_val)
self.emit(f"{wrapper_name} () {{")
with self.indented():
if is_method:
self.emit('local this="$1"')
self.emit('shift')
self.emit(f"local __attempts={attempts}")
self.emit(f"local __delay={delay}")
self.emit("local __i")
self.emit("for __i in $(seq 1 $__attempts); do")
with self.indented():
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(params))])
if is_method:
self.emit(f'if {wrapped_name} "$this" {params_str}; then')
else:
self.emit(f'if {wrapped_name} {params_str}; then')
with self.indented():
self.emit("return 0")
self.emit("fi")
self.emit('sleep "$__delay"')
self.emit("done")
self.emit("return 1")
self.emit("}")
self.emit()
def _generate_log_wrapper(self, wrapped_name: str, wrapper_name: str,
params: List[Parameter], is_method: bool):
self.emit(f"{wrapper_name} () {{")
with self.indented():
if is_method:
self.emit('local this="$1"')
self.emit('shift')
self.emit(f'echo "[LOG] Calling {wrapped_name}" >&2')
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(params))])
if is_method:
self.emit(f'{wrapped_name} "$this" {params_str}')
else:
self.emit(f'{wrapped_name} {params_str}')
self.emit('local __ret=$?')
self.emit(f'echo "[LOG] {wrapped_name} returned $__ret" >&2')
self.emit('return $__ret')
self.emit("}")
self.emit()
def _generate_cache_wrapper(self, decorator: Decorator, wrapped_name: str,
wrapper_name: str, params: List[Parameter], is_method: bool):
ttl = 60
for arg_name, arg_val in decorator.arguments:
if arg_name == "ttl":
ttl = self.generate_expr(arg_val)
self.emit(f"declare -gA __ct_cache_{wrapper_name}=()")
self.emit(f"declare -g __ct_cache_time_{wrapper_name}=0")
self.emit()
self.emit(f"{wrapper_name} () {{")
with self.indented():
if is_method:
self.emit('local this="$1"')
self.emit('shift')
self.emit(f'local __key="$this:$*"')
else:
self.emit(f'local __key="$*"')
self.emit(f'local __now=$(date +%s)')
self.emit(f'local __cache_age=$((__now - __ct_cache_time_{wrapper_name}))')
self.emit(f'if [[ $__cache_age -lt {ttl} ]] && [[ -n "${{__ct_cache_{wrapper_name}[$__key]:-}}" ]]; then')
with self.indented():
self.emit(f'{RET_VAR}="${{__ct_cache_{wrapper_name}[$__key]}}"')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
self.emit("fi")
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(params))])
if is_method:
self.emit(f'local __result=$({wrapped_name} "$this" {params_str})')
else:
self.emit(f'local __result=$({wrapped_name} {params_str})')
self.emit(f'{RET_VAR}="$__result"')
self.emit(f'__ct_cache_{wrapper_name}["$__key"]="$__result"')
self.emit(f'__ct_cache_time_{wrapper_name}=$__now')
self.emit('echo "$__result"')
self.emit("}")
self.emit()
def _generate_passthrough_wrapper(self, wrapped_name: str, wrapper_name: str,
params: List[Parameter], is_method: bool):
self.emit(f"{wrapper_name} () {{")
with self.indented():
if is_method:
self.emit('local this="$1"')
self.emit('shift')
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(params))])
if is_method:
self.emit(f'{wrapped_name} "$this" {params_str}')
else:
self.emit(f'{wrapped_name} {params_str}')
self.emit("}")
self.emit()
def _generate_validate_wrapper(self, decorator: Decorator, wrapped_name: str,
wrapper_name: str, params: List[Parameter], is_method: bool):
validations = {}
for arg_name, arg_val in decorator.arguments:
if arg_name and hasattr(arg_val, 'value'):
validations[arg_name] = arg_val.value
self.emit(f"{wrapper_name} () {{")
with self.indented():
if is_method:
self.emit('local this="$1"')
self.emit('shift')
for i, param in enumerate(params):
rule = validations.get(param.name)
if rule:
self._emit_validation_check(param.name, rule, i + 1)
params_str = " ".join([f'"${{{i + 1}}}"' for i in range(len(params))])
if is_method:
self.emit(f'{wrapped_name} "$this" {params_str}')
else:
self.emit(f'{wrapped_name} {params_str}')
self.emit("}")
self.emit()
def _emit_validation_check(self, param_name: str, rule: str, arg_pos: int):
self.emit(f'local {param_name}="${{{arg_pos}}}"')
if "int" in rule:
self.emit(f'if ! [[ "${param_name}" =~ ^-?[0-9]+$ ]]; then')
with self.indented():
self.emit(f'echo "Validation error: {param_name} must be integer" >&2')
self.emit('return 1')
self.emit('fi')
for match in re.finditer(r'(>=|<=|>|<|==|!=)\s*(-?\d+)', rule):
op, val = match.groups()
bash_op = {'>': '-gt', '<': '-lt', '>=': '-ge', '<=': '-le', '==': '-eq', '!=': '-ne'}[op]
self.emit(f'if ! [[ ${param_name} {bash_op} {val} ]]; then')
with self.indented():
self.emit(f'echo "Validation error: {param_name} must be {op} {val}" >&2')
self.emit('return 1')
self.emit('fi')
elif "string" in rule:
if "nonempty" in rule or "required" in rule:
self.emit(f'if [[ -z "${param_name}" ]]; then')
with self.indented():
self.emit(f'echo "Validation error: {param_name} cannot be empty" >&2')
self.emit('return 1')
self.emit('fi')
bootstrap/dispatch_codegen.py deleted 100644 → 0
View file @ 5f7339cd
from .ast_nodes import (
CallExpr, MemberAccess, Identifier, ThisExpr, Assignment, ArrayLiteral,
DictLiteral, NewExpr, AsyncExpr, Lambda, ExpressionStmt, BaseCall, ReturnStmt,
StringLiteral, BinaryOp, IndexAccess, TypeAnnotation, IntegerLiteral,
FloatLiteral, BoolLiteral
)
from .methods import (
STRING_METHODS, ARRAY_METHODS, DICT_METHODS, FILE_HANDLE_METHODS,
PROCESS_HANDLE_METHODS,
NAMESPACE_METHODS, BUILTIN_NAMESPACES, BUILTIN_FUNCS, get_method_names
)
from .constants import RET_VAR, RET_ARR, COPROC_PREFIX
ARR_METHODS = {name: m.bash_func for name, m in ARRAY_METHODS.items()}
STR_METHODS = {name: m.bash_func for name, m in STRING_METHODS.items()}
DICT_METHODS_MAP = {name: m.bash_func for name, m in DICT_METHODS.items()}
FILE_HANDLE_METHODS_MAP = {name: m.bash_func for name, m in FILE_HANDLE_METHODS.items()}
FILE_HANDLE_METHODS_MAP["__enter__"] = "__ct_fh___enter__"
FILE_HANDLE_METHODS_MAP["__exit__"] = "__ct_fh___exit__"
class DispatchMixin:
"""Mixin for method dispatch and assignment."""
def _report_unknown_method(self, obj_type: str, method: str, available: set, location=None):
"""Report unknown method error."""
from .errors import CompileError
available_str = ", ".join(sorted(available))
msg = f"Unknown method '{method}' for type '{obj_type}'. Available: {available_str}"
filename = location.filename if location else "<unknown>"
line = location.line if location else 0
col = location.column if location else 0
self.errors.add(CompileError(msg, filename, line, col))
def _validate_namespace_method(self, namespace: str, method: str, location=None) -> bool:
"""Validate that method exists for namespace. Returns True if valid."""
if namespace not in NAMESPACE_METHODS:
return True
methods = NAMESPACE_METHODS[namespace]
if method not in methods:
self._report_unknown_method(namespace, method, methods, location)
return False
return True
def _validate_type_method(self, var_type: str, method: str, location=None) -> bool:
"""Validate that method exists for variable type. Returns True if valid."""
available = get_method_names(var_type)
if not available:
return True
if method not in available:
self._report_unknown_method(var_type, method, available, location)
return False
return True
def _emit_array_assign(self, target: str):
"""Emit array assignment from RET_VAR."""
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -a {target}=("${{{RET_VAR}[@]}}")')
else:
self.emit(f'{target}=("${{{RET_VAR}[@]}}")')
def _is_method_call_with_side_effects(self, expr) -> bool:
"""Check if expression is a method call that may have side effects."""
if isinstance(expr, CallExpr):
if isinstance(expr.callee, MemberAccess):
if isinstance(expr.callee.object, (ThisExpr, Identifier)):
return True
return False
def _generate_call_arg(self, arg) -> str:
"""Generate a single call argument, handling arrays and callbacks specially."""
if isinstance(arg, Identifier):
name = arg.name
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(name, name)
if mapped_name in self.array_vars or name in self.array_vars:
return mapped_name
if mapped_name in self.dict_vars or name in self.dict_vars:
return mapped_name
if name in self.functions:
return name
if name in getattr(self, 'callback_vars', set()):
return f'${{{name}}}'
return self.generate_expr(arg)
def _generate_call_args_str(self, arguments: list) -> str:
"""Generate argument string for function/method calls."""
args = [self._generate_call_arg(arg) for arg in arguments]
return " ".join([f'"{a}"' for a in args])
def _emit_assign_with_op(self, target: str, value: str, operator: str):
"""Emit assignment with compound operator support."""
if operator == "=":
self.emit_var_assign(target, value)
elif operator == "+=":
self.emit(f'{target}=$((${{target}} + {value}))'.replace('target', target))
elif operator == "-=":
self.emit(f'{target}=$((${{target}} - {value}))'.replace('target', target))
elif operator == "*=":
self.emit(f'{target}=$((${{target}} * {value}))'.replace('target', target))
elif operator == "/=":
self.emit(f'{target}=$((${{target}} / {value}))'.replace('target', target))
def _infer_expr_type(self, expr) -> str:
if isinstance(expr, IntegerLiteral):
return "int"
elif isinstance(expr, FloatLiteral):
return "float"
elif isinstance(expr, StringLiteral):
return "string"
elif isinstance(expr, BoolLiteral):
return "bool"
elif isinstance(expr, ArrayLiteral):
return "array"
elif isinstance(expr, DictLiteral):
return "dict"
elif isinstance(expr, NewExpr):
return expr.class_name
elif isinstance(expr, Lambda):
return "func"
elif isinstance(expr, MemberAccess):
if isinstance(expr.object, Identifier) and expr.object.name == "env":
return "any"
elif isinstance(expr, Identifier):
name = expr.name
if name in self.array_vars:
return "array"
elif name in self.dict_vars:
return "dict"
elif name in self.instance_vars:
return self.instance_vars[name]
elif name in self.callback_vars:
return "func"
return "any"
def _check_type_compatibility(self, expected: TypeAnnotation, actual_type: str, location) -> bool:
expected_name = expected.name
if expected.is_array:
expected_name = "array"
if expected_name == "any" or actual_type == "any":
return True
if expected_name == actual_type:
return True
if expected_name in ("int", "float") and actual_type in ("int", "float"):
return True
if expected_name in self.classes and actual_type in self.classes:
return True
return False
def _apply_type_annotation(self, target: str, type_annotation: TypeAnnotation):
if type_annotation.name == "array" or type_annotation.is_array:
self.array_vars.add(target)
elif type_annotation.name == "dict":
self.dict_vars.add(target)
elif type_annotation.name in self.classes:
self.object_vars.add(target)
self.instance_vars[target] = type_annotation.name
elif type_annotation.name == "func":
self.callback_vars.add(target)
def generate_assignment(self, stmt: Assignment):
if isinstance(stmt.target, MemberAccess):
if isinstance(stmt.target.object, ThisExpr):
self._generate_this_field_assignment(stmt)
return
if isinstance(stmt.target.object, Identifier) and stmt.target.object.name == "env":
var_name = stmt.target.member
value = self.generate_expr(stmt.value)
self.emit(f'export {var_name}="{value}"')
return
if isinstance(stmt.target.object, Identifier):
obj_name = stmt.target.object.name
if obj_name in self.object_vars:
self._generate_obj_field_assignment(stmt, obj_name)
return
if obj_name in self.dict_vars:
self._generate_dict_field_assignment(stmt, obj_name)
return
target = self.generate_lvalue(stmt.target)
if stmt.type_annotation:
self._apply_type_annotation(target, stmt.type_annotation)
if self.type_check and stmt.value:
actual_type = self._infer_expr_type(stmt.value)
if not self._check_type_compatibility(stmt.type_annotation, actual_type, stmt.location):
expected = stmt.type_annotation.name
if stmt.type_annotation.is_array:
expected = f"{expected}[]"
msg = f"Type mismatch: expected '{expected}', got '{actual_type}'"
if self.warn_types:
import sys
loc = stmt.location
print(f"Warning: {msg} at {loc.filename}:{loc.line}:{loc.column}", file=sys.stderr)
else:
self.errors.add_error(
message=msg,
filename=stmt.location.filename if stmt.location else "<unknown>",
line=stmt.location.line if stmt.location else 0,
column=stmt.location.column if stmt.location else 0
)
if isinstance(stmt.value, AsyncExpr):
self._generate_async_assignment(stmt, target)
return
if isinstance(stmt.value, BinaryOp) and stmt.value.operator == "|":
self._generate_pipe_assignment(stmt, target)
return
if isinstance(stmt.value, Lambda):
self.generate_lambda_as_function(stmt.value, target)
self.emit_var_assign(target, target)
self.callback_vars.add(target)
return
if isinstance(stmt.value, NewExpr):
resolved_class = self._resolve_name(stmt.value.class_name)
args_str = self._generate_call_args_str(stmt.value.arguments)
self.emit(f'{resolved_class} {args_str}')
self.emit_var_assign(target, '$__ct_last_instance')
self.object_vars.add(target)
self.instance_vars[target] = resolved_class
return
if isinstance(stmt.value, CallExpr) and isinstance(stmt.value.callee, Identifier):
callee_name = stmt.value.callee.name
resolved_callee = self._resolve_name(callee_name)
if resolved_callee in self.classes:
args_str = self._generate_call_args_str(stmt.value.arguments)
self.emit(f'{resolved_callee} {args_str}')
self.emit_var_assign(target, '$__ct_last_instance')
self.object_vars.add(target)
self.instance_vars[target] = resolved_callee
return
if isinstance(stmt.value, CallExpr) and isinstance(stmt.value.callee, MemberAccess):
callee = stmt.value.callee
if isinstance(callee.object, Identifier) and callee.object.name == "fs" and callee.member == "open":
args_str = self._generate_call_args_str(stmt.value.arguments)
self.emit(f'__ct_fs_open {args_str} >/dev/null')
self.emit_var_assign(target, f'${RET_VAR}')
self.file_handle_vars.add(target)
return
if isinstance(callee.object, Identifier) and callee.object.name == "json" and callee.member == "parse":
args = [self.generate_expr(arg) for arg in stmt.value.arguments]
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -A {target}=()')
self.emit(f'__ct_json_parse "{args[0]}" "{target}"')
self.dict_vars.add(target)
return
if isinstance(callee.object, Identifier) and callee.object.name == "json" and callee.member == "unmarshal":
args = [self.generate_expr(arg) for arg in stmt.value.arguments]
class_name = stmt.value.arguments[1].name if isinstance(stmt.value.arguments[1], Identifier) else args[1]
self.emit(f'__ct_json_unmarshal "{args[0]}" "{class_name}"')
self.emit_var_assign(target, '$__ct_last_instance')
self.object_vars.add(target)
self.instance_vars[target] = class_name
return
if isinstance(callee.object, Identifier) and callee.object.name == "reflect" and callee.member == "create":
args = [self.generate_expr(arg) for arg in stmt.value.arguments]
self.emit(f'__ct_reflect_create "{args[0]}"')
self.emit_var_assign(target, '$__ct_last_instance')
self.object_vars.add(target)
return
if isinstance(callee.object, Identifier) and callee.object.name == "reflect" and callee.member == "fields":
args = [self._generate_call_arg(arg) for arg in stmt.value.arguments]
self.emit(f'__ct_reflect_fields "{args[0]}"')
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -a {target}=("${{{RET_ARR}[@]}}")')
else:
self.emit(f'{target}=("${{{RET_ARR}[@]}}")')
self.array_vars.add(target)
return
if self._generate_method_call_assignment(stmt, target):
return
if isinstance(stmt.value, CallExpr) and isinstance(stmt.value.callee, Identifier):
func_name = stmt.value.callee.name
resolved_func = self._resolve_name(func_name)
if func_name not in BUILTIN_FUNCS and resolved_func not in self.classes:
args_str = self._generate_call_args_str(stmt.value.arguments)
if func_name in self.callback_vars:
self.emit_var_assign(target, f'$("${{{func_name}}}" {args_str})')
elif resolved_func in self.functions:
self.emit(f'{resolved_func} {args_str} >/dev/null')
self.emit_var_assign(target, f'${RET_VAR}')
else:
self.emit_var_assign(target, f'$({func_name} {args_str})')
return
if isinstance(stmt.value, ArrayLiteral):
self._generate_array_assignment(stmt, target)
return
if isinstance(stmt.value, DictLiteral):
self._generate_dict_assignment(stmt, target)
return
if isinstance(stmt.value, Identifier):
src_name = stmt.value.name
if src_name in self.array_vars:
self.emit(f'{target}=("${{{src_name}[@]}}")')
self.array_vars.add(target)
return
if isinstance(stmt.value, MemberAccess):
field_name = stmt.value.member
is_array_field = any(
self.class_field_types.get((cls, field_name)) == "array"
for cls in self.classes
)
is_dict_field = any(
self.class_field_types.get((cls, field_name)) == "dict"
for cls in self.classes
)
if is_array_field or is_dict_field:
obj = self.generate_expr(stmt.value.object)
if obj.startswith('${') and obj.endswith('}'):
var_name = obj[2:-1]
arr_ref = f'${{{var_name}}}_{field_name}'
else:
arr_ref = f'{obj}_{field_name}'
if self.in_function:
if target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -n {target}="{arr_ref}"')
else:
self.emit(f'declare -gn {target}="{arr_ref}"')
if is_array_field:
self.array_vars.add(target)
else:
self.dict_vars.add(target)
return
is_object_field = any(
self.class_field_types.get((cls, field_name)) == "object"
for cls in self.classes
)
if is_object_field:
obj_class = None
if isinstance(stmt.value.object, Identifier) and stmt.value.object.name in self.instance_vars:
parent_class = self.instance_vars[stmt.value.object.name]
obj_class = self.class_field_class.get((parent_class, field_name))
if not obj_class:
for cls in self.classes:
obj_class = self.class_field_class.get((cls, field_name))
if obj_class:
break
value = self.generate_expr(stmt.value)
self.emit_var_assign(target, value)
self.object_vars.add(target)
if obj_class:
self.instance_vars[target] = obj_class
return
if isinstance(stmt.value, BinaryOp) and stmt.value.operator in ("==", "!=", "<", ">", "<=", ">=", "&&", "||"):
cond = self.generate_condition(stmt.value)
if self.in_function and target not in self.local_vars and target not in self.global_vars and '.' not in target and '[' not in target:
self.local_vars.add(target)
self.emit(f'local {target}')
self.emit(f'if {cond}; then {target}="true"; else {target}="false"; fi')
return
value = self.generate_expr(stmt.value)
if stmt.operator == "=":
self.emit_var_assign(target, value)
elif stmt.operator == "+=":
self.emit('{}=$((${{{}}} + {}))'.format(target, target, value))
elif stmt.operator == "-=":
self.emit('{}=$((${{{}}} - {}))'.format(target, target, value))
elif stmt.operator == "*=":
self.emit('{}=$((${{{}}} * {}))'.format(target, target, value))
elif stmt.operator == "/=":
self.emit('{}=$((${{{}}} / {}))'.format(target, target, value))
def _generate_this_field_assignment(self, stmt: Assignment):
"""Generate this.field = value assignment."""
field = stmt.target.member
field_type = self.class_field_types.get((self.current_class, field)) if self.current_class else None
if field_type == "array" and isinstance(stmt.value, Identifier):
src_name = stmt.value.name
if src_name in self.current_param_positions:
pos = self.current_param_positions[src_name]
self.emit(f'local -n __ct_src_{field}="${{{pos}}}"')
else:
self.emit(f'local -n __ct_src_{field}="${{{src_name}}}"')
self.emit(f'local -n __ct_dst_{field}="${{this}}_{field}"')
self.emit(f'__ct_dst_{field}=("${{__ct_src_{field}[@]}}")')
return
if isinstance(stmt.value, NewExpr):
resolved_class = self._resolve_name(stmt.value.class_name)
args_str = self._generate_call_args_str(stmt.value.arguments)
self.emit(f'{resolved_class} {args_str}')
self.emit(f'local __ct_tmp_{field}="$__ct_last_instance"')
self.emit(f'__CT_OBJ["$this.{field}"]="$__ct_tmp_{field}"')
return
if isinstance(stmt.value, CallExpr) and isinstance(stmt.value.callee, Identifier):
callee_name = stmt.value.callee.name
if callee_name in self.classes:
args_str = self._generate_call_args_str(stmt.value.arguments)
self.emit(f'{callee_name} {args_str}')
self.emit(f'local __ct_tmp_{field}="$__ct_last_instance"')
self.emit(f'__CT_OBJ["$this.{field}"]="$__ct_tmp_{field}"')
return
value = self.generate_expr(stmt.value)
if stmt.operator == "=":
self.emit(f'__CT_OBJ["$this.{field}"]="{value}"')
elif stmt.operator == "+=":
self.emit(f'__CT_OBJ["$this.{field}"]="$(( ${{__CT_OBJ["$this.{field}"]}} + {value} ))"')
elif stmt.operator == "-=":
self.emit(f'__CT_OBJ["$this.{field}"]="$(( ${{__CT_OBJ["$this.{field}"]}} - {value} ))"')
elif stmt.operator == "*=":
self.emit(f'__CT_OBJ["$this.{field}"]="$(( ${{__CT_OBJ["$this.{field}"]}} * {value} ))"')
elif stmt.operator == "/=":
self.emit(f'__CT_OBJ["$this.{field}"]="$(( ${{__CT_OBJ["$this.{field}"]}} / {value} ))"')
elif stmt.operator == "..=":
self.emit(f'__CT_OBJ["$this.{field}"]="${{__CT_OBJ["$this.{field}"]}}{value}"')
else:
self.emit(f'__CT_OBJ["$this.{field}"]="{value}"')
def _generate_obj_field_assignment(self, stmt: Assignment, obj_name: str):
"""Generate obj.field = value assignment for object variables."""
field = stmt.target.member
value = self.generate_expr(stmt.value)
if stmt.operator == "=":
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="{value}"')
elif stmt.operator == "+=":
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="$(( ${{__CT_OBJ["${{{obj_name}}}.{field}"]}} + {value} ))"')
elif stmt.operator == "-=":
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="$(( ${{__CT_OBJ["${{{obj_name}}}.{field}"]}} - {value} ))"')
elif stmt.operator == "*=":
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="$(( ${{__CT_OBJ["${{{obj_name}}}.{field}"]}} * {value} ))"')
elif stmt.operator == "/=":
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="$(( ${{__CT_OBJ["${{{obj_name}}}.{field}"]}} / {value} ))"')
elif stmt.operator == "..=":
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="${{__CT_OBJ["${{{obj_name}}}.{field}"]}}{value}"')
else:
self.emit(f'__CT_OBJ["${{{obj_name}}}.{field}"]="{value}"')
def _generate_dict_field_assignment(self, stmt: Assignment, dict_name: str):
"""Generate dict.field = value assignment for dict variables."""
field = stmt.target.member
value = self.generate_expr(stmt.value)
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(dict_name, dict_name)
if stmt.operator == "=":
self.emit(f'{mapped_name}["{field}"]="{value}"')
elif stmt.operator == "+=":
self.emit(f'{mapped_name}["{field}"]="$(( ${{{mapped_name}["{field}"]}} + {value} ))"')
elif stmt.operator == "-=":
self.emit(f'{mapped_name}["{field}"]="$(( ${{{mapped_name}["{field}"]}} - {value} ))"')
elif stmt.operator == "..=":
self.emit(f'{mapped_name}["{field}"]="${{{mapped_name}["{field}"]}}{value}"')
else:
self.emit(f'{mapped_name}["{field}"]="{value}"')
def _generate_method_call_assignment(self, stmt: Assignment, target: str) -> bool:
"""Generate method call assignment. Returns True if handled."""
callee = stmt.value.callee
method = callee.member if isinstance(callee, MemberAccess) else None
location = getattr(stmt.value, 'location', None) or getattr(callee, 'location', None)
if method in ("map", "filter"):
args = [self._generate_call_arg(arg) for arg in stmt.value.arguments if not isinstance(arg, Lambda)]
else:
args = [self._generate_call_arg(arg) for arg in stmt.value.arguments]
args_str = " ".join([f'"{a}"' for a in args])
if isinstance(callee.object, ThisExpr) and self.current_class:
method = callee.member
self.emit(f'__ct_class_{self.current_class}_{method} "$this" {args_str} >/dev/null')
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
if isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, ThisExpr):
field_name = callee.object.member
method = callee.member
field_type = self.class_field_types.get((self.current_class, field_name)) if self.current_class else None
if field_type == "array":
if method in ARR_METHODS:
arr_name = f'"${{this}}_{field_name}"'
self.emit(f'{ARR_METHODS[method]} {arr_name} {args_str} >/dev/null'.strip())
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
else:
self._validate_type_method("array", method, location)
if field_type == "dict":
if method in DICT_METHODS:
dict_ref = f'"${{this}}_{field_name}"'
self.emit(f'{DICT_METHODS_MAP[method]} {dict_ref} {args_str} >/dev/null'.strip())
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
else:
self._validate_type_method("dict", method, location)
if field_type == "object":
obj_ref = f'${{__CT_OBJ["$this.{field_name}"]}}'
self.emit(f'__ct_call_method "{obj_ref}" "{method}" {args_str} >/dev/null')
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
return False
if isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, Identifier):
var_name = callee.object.object.name
field_name = callee.object.member
method = callee.member
is_array_field = any(
self.class_field_types.get((cls, field_name)) == "array"
for cls in self.classes
)
is_dict_field = any(
self.class_field_types.get((cls, field_name)) == "dict"
for cls in self.classes
)
if is_array_field and method in ARR_METHODS:
arr_ref = f'"${{{var_name}}}_{field_name}"'
self.emit(f'{ARR_METHODS[method]} {arr_ref} {args_str} >/dev/null'.strip())
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
if is_dict_field and method in DICT_METHODS_MAP:
dict_ref = f'"${{{var_name}}}_{field_name}"'
self.emit(f'{DICT_METHODS_MAP[method]} {dict_ref} {args_str} >/dev/null'.strip())
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
field_ref = f'${{__CT_OBJ["${{{var_name}}}.{field_name}"]:-}}'
if method in STR_METHODS:
self.emit(f'{STR_METHODS[method]} "{field_ref}" {args_str} >/dev/null'.replace(' ', ' '))
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
if isinstance(callee.object, Identifier):
obj_name = callee.object.name
method = callee.member
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(obj_name, obj_name)
resolved = self._resolve_qualified(obj_name, method)
if resolved is not None:
if resolved in self.classes:
self.emit(f'{resolved} {args_str}')
self.emit_var_assign(target, '$__ct_last_instance')
self.object_vars.add(target)
self.instance_vars[target] = resolved
elif resolved in self.functions:
self.emit(f'{resolved} {args_str} >/dev/null')
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
else:
self.emit_var_assign(target, f'$({resolved} {args_str})')
return True
if obj_name in self.busing_names:
self.emit_var_assign(target, f'$({method} {args_str})')
return True
if obj_name in BUILTIN_NAMESPACES:
self._validate_namespace_method(obj_name, method, location)
return False
if mapped_name in self.array_vars:
if method in ARR_METHODS:
func_name = ARR_METHODS[method]
arr_ref = mapped_name
if method == "push" and len(args) == 1:
self.emit(f'{mapped_name}+=("{args[0]}")')
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
elif method in ("map", "filter") and len(stmt.value.arguments) >= 1:
first_arg = stmt.value.arguments[0]
if isinstance(first_arg, Lambda):
lambda_name = self.generate_lambda(first_arg)
self.emit(f'{func_name} "{arr_ref}" "{lambda_name}"')
else:
self.emit(f'{func_name} "{arr_ref}" {args_str}'.replace(' ', ' '))
self.array_vars.add(target)
self._emit_array_assign(target)
elif method == "slice":
self.emit(f'{func_name} "{arr_ref}" {args_str}')
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -a {target}=("${{{RET_ARR}[@]}}")')
else:
self.emit(f'{target}=("${{{RET_ARR}[@]}}")')
self.array_vars.add(target)
return True
else:
self.emit(f'{func_name} "{arr_ref}" {args_str} >/dev/null'.replace(' ', ' '))
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
else:
self._validate_type_method("array", method, location)
if mapped_name in self.dict_vars:
if method == "keys":
dict_ref = mapped_name
self.emit(f'__ct_dict_keys "{dict_ref}"')
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -a {target}=("${{{RET_ARR}[@]}}")')
else:
self.emit(f'{target}=("${{{RET_ARR}[@]}}")')
self.array_vars.add(target)
return True
elif method in DICT_METHODS:
func_name = DICT_METHODS_MAP[method]
dict_ref = mapped_name
self.emit(f'{func_name} "{dict_ref}" {args_str} >/dev/null'.replace(' ', ' '))
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
else:
self._validate_type_method("dict", method, location)
if obj_name in self.file_handle_vars:
if method in FILE_HANDLE_METHODS:
func_name = FILE_HANDLE_METHODS_MAP[method]
self.emit(f'{func_name} "${obj_name}" {args_str} >/dev/null'.replace(' ', ' '))
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
else:
self._validate_type_method("file_handle", method, location)
if obj_name in self.process_handle_vars:
if method in PROCESS_HANDLE_METHODS:
self._generate_process_method(obj_name, method, args, target)
return True
else:
self._validate_type_method("process_handle", method, location)
if obj_name in self.object_vars:
ret_type = self._get_method_return_type(obj_name, method)
obj = self.generate_expr(callee.object)
self.emit(f'__ct_call_method "{obj}" "{method}" {args_str} >/dev/null')
if ret_type == "array":
self.emit_var_assign(target, f'${RET_VAR}')
self.array_vars.add(target)
self.nameref_vars.add(target)
elif ret_type == "dict":
self.emit_var_assign(target, f'${RET_VAR}')
self.dict_vars.add(target)
self.nameref_vars.add(target)
elif ret_type == "object":
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
self.object_vars.add(target)
else:
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
if method in STR_METHODS:
obj = self.generate_expr(callee.object)
func_name = STR_METHODS[method]
if method == "split":
self.emit(f'{func_name} "{obj}" {args_str}')
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
self.emit(f'local -a {target}=("${{{RET_ARR}[@]}}")')
else:
self.emit(f'{target}=("${{{RET_ARR}[@]}}")')
self.array_vars.add(target)
return True
self.emit(f'{func_name} "{obj}" {args_str} >/dev/null'.replace(' ', ' '))
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
if method in DICT_METHODS:
obj = self.generate_expr(callee.object)
func_name = DICT_METHODS_MAP[method]
self.emit(f'{func_name} "{obj}" {args_str} >/dev/null'.replace(' ', ' '))
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
ret_type = self._get_method_return_type(obj_name, method)
obj = self.generate_expr(callee.object)
self.emit(f'__ct_call_method "{obj}" "{method}" {args_str} >/dev/null')
if ret_type == "array":
self.emit_var_assign(target, f'${RET_VAR}')
self.array_vars.add(target)
self.nameref_vars.add(target)
elif ret_type == "dict":
self.emit_var_assign(target, f'${RET_VAR}')
self.dict_vars.add(target)
self.nameref_vars.add(target)
elif ret_type == "object":
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
self.object_vars.add(target)
else:
self._emit_assign_with_op(target, f'${RET_VAR}', stmt.operator)
return True
return False
def _get_method_return_type(self, obj_name: str, method_name: str) -> str:
"""Analyze method to determine return type (array, dict, object, or None)."""
class_name = self.instance_vars.get(obj_name)
if not class_name or class_name not in self.classes:
return None
class_decl = self.classes[class_name]
for method in class_decl.methods:
if method.name == method_name:
return self._analyze_return_type(method.body, class_name)
return None
def _analyze_return_type(self, body, class_name: str) -> str:
"""Analyze function body to determine what type it returns."""
if not body or not hasattr(body, 'statements'):
return None
for stmt in body.statements:
if isinstance(stmt, ReturnStmt) and stmt.value:
if isinstance(stmt.value, MemberAccess) and isinstance(stmt.value.object, ThisExpr):
field_name = stmt.value.member
field_type = self.class_field_types.get((class_name, field_name))
return field_type
if isinstance(stmt.value, ArrayLiteral):
return "array"
if isinstance(stmt.value, DictLiteral):
return "dict"
if isinstance(stmt.value, NewExpr):
return "object"
if isinstance(stmt.value, CallExpr) and isinstance(stmt.value.callee, Identifier):
if stmt.value.callee.name in self.classes:
return "object"
return None
def _generate_array_assignment(self, stmt: Assignment, target: str):
"""Generate array literal assignment."""
elements = [self.generate_expr(e) for e in stmt.value.elements]
arr_content = " ".join([f'"{e}"' for e in elements]) if elements else ""
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
if arr_content:
self.emit(f'local -a {target}=({arr_content})')
else:
self.emit(f'local -a {target}=()')
elif self.in_function and target in self.global_vars:
if arr_content:
self.emit(f'{target}=({arr_content})')
else:
self.emit(f'{target}=()')
else:
if arr_content:
self.emit(f'declare -ga {target}=({arr_content})')
else:
self.emit(f'declare -ga {target}=()')
self.array_vars.add(target)
def _generate_dict_assignment(self, stmt: Assignment, target: str):
"""Generate dict literal assignment."""
pairs = []
for k, v in stmt.value.pairs:
key = self.generate_expr(k)
val = self.generate_expr(v)
pairs.append(f'[{key}]="{val}"')
dict_content = " ".join(pairs) if pairs else ""
if self.in_function and target not in self.local_vars and target not in self.global_vars:
self.local_vars.add(target)
if dict_content:
self.emit(f'local -A {target}=({dict_content})')
else:
self.emit(f'local -A {target}=()')
else:
if dict_content:
self.emit(f'declare -gA {target}=({dict_content})')
else:
self.emit(f'declare -gA {target}=()')
self.dict_vars.add(target)
def _generate_async_assignment(self, stmt: Assignment, target: str):
self.coproc_counter += 1
cp_name = f'{COPROC_PREFIX}{self.coproc_counter}'
inner = stmt.value.expression
if isinstance(inner, CallExpr) and self._is_shell_command(inner):
cmd_str = self._extract_shell_command_str(inner)
elif isinstance(inner, CallExpr) and isinstance(inner.callee, Identifier):
func_name = inner.callee.name
args_str = self._generate_call_args_str(inner.arguments)
cmd_str = f'exec 3>&1; {func_name} {args_str}'.strip()
else:
cmd_str = self.generate_expr(inner)
self.emit(f'coproc {cp_name} {{ {cmd_str}; }}')
self.emit(f'exec {{{cp_name}_wr}}>&${{{cp_name}[1]}}')
self.emit(f'exec {{{cp_name}_rd}}<&${{{cp_name}[0]}}')
self.emit(f'{cp_name}_pid=${cp_name}_PID')
self.emit(f'eval "exec ${{{cp_name}[1]}}>&-"')
self.process_handle_vars.add(target)
self.process_handle_map[target] = cp_name
def _generate_process_method(self, var_name: str, method: str, args: list, target: str = None):
cp = self.process_handle_map[var_name]
if method == "write":
data = args[0] if args else ""
self.emit(f'echo "{data}" >&${cp}_wr')
elif method == "read":
self.emit(f'read -r {RET_VAR} <&${cp}_rd')
if target:
self.emit_var_assign(target, f'${RET_VAR}')
else:
self.emit(f'echo "${RET_VAR}"')
elif method == "close":
self.emit(f'exec {{{cp}_wr}}>&-')
elif method == "kill":
self.emit(f'kill ${cp}_pid 2>/dev/null || true')
elif method == "wait":
self.emit(f'wait ${cp}_pid 2>/dev/null || true')
def _generate_pipe_assignment(self, stmt: Assignment, target: str):
"""Generate pipe expression assignment."""
elements = self._collect_pipe_chain(stmt.value)
if all(self._is_shell_command(e) for e in elements):
commands = [self._extract_shell_command_str(e) for e in elements]
pipe_cmd = " | ".join(commands)
self.emit_var_assign(target, f'$({pipe_cmd})')
return
if any(self._is_shell_command(e) for e in elements[1:]):
self._generate_mixed_pipe_assignment(elements, target)
return
self._generate_functional_pipe_assignment(elements, target)
def _generate_functional_pipe_assignment(self, elements: list, target: str):
"""Generate functional pipe with proper variable handling."""
if not elements:
return
first = elements[0]
if isinstance(first, CallExpr):
if self._is_shell_command(first):
cmd = self._extract_shell_command_str(first)
current_var = self.new_temp()
self.emit_var_assign(current_var, f'$({cmd})')
else:
call_code = self.generate_call_statement(first)
self.emit(f'{call_code} >/dev/null')
current_var = self.new_temp()
self.emit_var_assign(current_var, f'${RET_VAR}')
elif isinstance(first, Identifier):
current_var = first.name
else:
current_var = self.new_temp()
self.emit_var_assign(current_var, self.generate_expr(first))
for elem in elements[1:]:
if isinstance(elem, CallExpr):
if self._is_shell_command(elem):
cmd = self._extract_shell_command_str(elem)
new_var = self.new_temp()
self.emit_var_assign(new_var, f'$(echo "${{{current_var}}}" | {cmd})')
current_var = new_var
else:
func_name = self._get_call_func_name(elem)
args_str = self._generate_call_args_str(elem.arguments)
if args_str:
self.emit(f'{func_name} "${{{current_var}}}" {args_str} >/dev/null')
else:
self.emit(f'{func_name} "${{{current_var}}}" >/dev/null')
new_var = self.new_temp()
self.emit_var_assign(new_var, f'${RET_VAR}')
current_var = new_var
elif isinstance(elem, Identifier):
self.emit(f'{elem.name} "${{{current_var}}}" >/dev/null')
new_var = self.new_temp()
self.emit_var_assign(new_var, f'${RET_VAR}')
current_var = new_var
self.emit_var_assign(target, f'${{{current_var}}}')
def _generate_mixed_pipe_assignment(self, elements: list, target: str):
"""Generate native bash pipe mixing CT functions and shell commands."""
parts = []
for elem in elements:
if self._is_shell_command(elem):
parts.append(self._extract_shell_command_str(elem))
elif isinstance(elem, CallExpr):
call_code = self.generate_call_statement(elem)
parts.append(f'{call_code} 3>&1')
elif isinstance(elem, Identifier):
parts.append(f'{elem.name} 3>&1')
else:
code = self.generate_expr(elem)
parts.append(code)
pipe_cmd = " | ".join(parts)
self.emit_var_assign(target, f'$({pipe_cmd})')
def _generate_mixed_pipe_stmt(self, elements: list):
"""Generate native bash pipe as statement (no assignment)."""
parts = []
for elem in elements:
if self._is_shell_command(elem):
parts.append(self._extract_shell_command_str(elem))
elif isinstance(elem, CallExpr):
call_code = self.generate_call_statement(elem)
parts.append(f'{call_code} 3>&1')
elif isinstance(elem, Identifier):
parts.append(f'{elem.name} 3>&1')
else:
code = self.generate_expr(elem)
parts.append(code)
pipe_cmd = " | ".join(parts)
self.emit(pipe_cmd)
def _get_call_func_name(self, call_expr: CallExpr) -> str:
"""Get function name from call expression."""
if isinstance(call_expr.callee, Identifier):
return call_expr.callee.name
elif isinstance(call_expr.callee, MemberAccess):
if isinstance(call_expr.callee.object, Identifier):
return f'{call_expr.callee.object.name}.{call_expr.callee.member}'
return "unknown"
def generate_expression_stmt(self, stmt: ExpressionStmt):
expr = stmt.expression
if isinstance(expr, BinaryOp) and expr.operator == "|":
elements = self._collect_pipe_chain(expr)
if all(self._is_shell_command(e) for e in elements):
commands = [self._extract_shell_command_str(e) for e in elements]
self.emit(" | ".join(commands))
return
if any(self._is_shell_command(e) for e in elements[1:]):
self._generate_mixed_pipe_stmt(elements)
return
if isinstance(expr, BaseCall):
if self.current_class:
parent_cls = self.classes.get(self.current_class)
if parent_cls and parent_cls.parent:
args_str = self._generate_call_args_str(expr.arguments)
self.emit(f'__ct_class_{parent_cls.parent}_construct "$this" {args_str}')
return
args_str = self._generate_call_args_str(expr.arguments)
self.emit(f'# base({args_str})')
return
if isinstance(expr, CallExpr):
if isinstance(expr.callee, Identifier) and expr.callee.name == "assert":
self._generate_assert_stmt(expr)
return
if isinstance(expr.callee, MemberAccess):
if self._handle_field_method_call(expr):
return
if isinstance(expr.callee.object, Identifier):
if self._handle_var_method_call(expr):
return
call_code = self.generate_call_statement(expr)
# Add >/dev/null to suppress return value echo for regular method calls
# But NOT for object method calls (this.method(), this.field.method(), obj.method())
# because they may print output that needs to pass through
if isinstance(expr.callee, MemberAccess):
obj = expr.callee.object
if isinstance(obj, Identifier):
if (obj.name not in BUILTIN_NAMESPACES and
obj.name not in self.object_vars and
obj.name not in getattr(self, 'busing_names', {})):
call_code = f'{call_code} >/dev/null'
# Don't add >/dev/null for ThisExpr or MemberAccess (this.field.method())
self.emit(call_code)
else:
result = self.generate_expr(expr)
if result:
self.emit(result)
def _generate_assert_stmt(self, expr: CallExpr):
"""Generate assert statement inline."""
condition_expr = expr.arguments[0] if expr.arguments else None
message = "Assertion failed"
if len(expr.arguments) >= 2:
msg_expr = expr.arguments[1]
if hasattr(msg_expr, 'value'):
message = msg_expr.value
else:
message = self.generate_expr(msg_expr)
if condition_expr:
cond = self.generate_condition(condition_expr)
escaped_msg = message.replace('"', '\\"')
self.emit(f'if {cond}; then :; else __ct_test_fail "{escaped_msg}"; return 1; fi')
def _handle_field_method_call(self, expr: CallExpr) -> bool:
"""Handle this.field.method() or var.field.method(). Returns True if handled."""
callee = expr.callee
args = [self._generate_call_arg(arg) for arg in expr.arguments]
if isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, ThisExpr):
field_name = callee.object.member
method = callee.member
return self._emit_field_method(f"${{this}}_{field_name}", method, args)
if isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, Identifier):
var_name = callee.object.object.name
field_name = callee.object.member
is_array_field = any(
self.class_field_types.get((cls, field_name)) == "array"
for cls in self.classes
)
is_dict_field = any(
self.class_field_types.get((cls, field_name)) == "dict"
for cls in self.classes
)
if is_array_field or is_dict_field:
return self._emit_field_method(f"${{{var_name}}}_{field_name}", expr.callee.member, args)
return False
def _emit_field_method(self, field_ref: str, method: str, args: list) -> bool:
"""Emit field method call."""
if method == "push":
self.emit(f'local -n __ct_tmp_arr="{field_ref}"')
for arg in args:
self.emit(f'__ct_tmp_arr+=("{arg}")')
return True
elif method == "pop":
self.emit(f'local -n __ct_tmp_arr="{field_ref}"')
self.emit("unset '__ct_tmp_arr[-1]'")
return True
elif method == "set" and len(args) >= 2:
self.emit(f'local -n __ct_tmp_arr="{field_ref}"')
self.emit(f'__ct_tmp_arr[{args[0]}]="{args[1]}"')
return True
elif method == "del" and len(args) >= 1:
self.emit(f'local -n __ct_tmp_arr="{field_ref}"')
self.emit(f"unset '__ct_tmp_arr[{args[0]}]'")
return True
elif method in ("len", "shift", "join", "get", "slice"):
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'__ct_arr_{method} "{field_ref}" {args_str} >/dev/null'.strip())
return True
elif method in ("has", "keys"):
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'__ct_dict_{method} "{field_ref}" {args_str} >/dev/null'.strip())
return True
return False
def _handle_var_method_call(self, expr: CallExpr) -> bool:
"""Handle var.method() calls. Returns True if handled."""
var_name = expr.callee.object.name
method = expr.callee.member
args = [self._generate_call_arg(arg) for arg in expr.arguments]
location = getattr(expr, 'location', None) or getattr(expr.callee, 'location', None)
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(var_name, var_name)
if mapped_name in self.array_vars:
arr_ref = mapped_name
if method == "push":
for i, arg_expr in enumerate(expr.arguments):
if self._is_method_call_with_side_effects(arg_expr):
call_code = self.generate_call_statement(arg_expr)
self.emit(f'{call_code} >/dev/null')
self.emit(f'{mapped_name}+=("$__CT_RET")')
else:
self.emit(f'{mapped_name}+=("{args[i]}")')
return True
elif method == "pop":
self.emit(f"unset '{mapped_name}[-1]'")
return True
elif method == "set" and len(args) >= 2:
self.emit(f'{mapped_name}[{args[0]}]="{args[1]}"')
return True
elif method in ("len", "shift", "join", "get", "slice", "map", "filter"):
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'__ct_arr_{method} "{arr_ref}" {args_str} >/dev/null'.strip())
return True
else:
self._validate_type_method("array", method, location)
return False
if mapped_name in self.dict_vars:
dict_ref = mapped_name
if method == "set" and len(args) >= 2:
self.emit(f'{mapped_name}[{args[0]}]="{args[1]}"')
return True
elif method == "del" and len(args) >= 1:
self.emit(f"unset '{mapped_name}[{args[0]}]'")
return True
elif method in ("get", "has", "keys"):
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'__ct_dict_{method} "{dict_ref}" {args_str} >/dev/null'.strip())
return True
else:
self._validate_type_method("dict", method, location)
return False
if var_name in self.file_handle_vars:
if method in FILE_HANDLE_METHODS:
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'{FILE_HANDLE_METHODS_MAP[method]} "${var_name}" {args_str}'.strip())
return True
else:
self._validate_type_method("file_handle", method, location)
return False
if var_name in self.process_handle_vars:
if method in PROCESS_HANDLE_METHODS:
self._generate_process_method(var_name, method, args)
return True
else:
self._validate_type_method("process_handle", method, location)
return False
if method in STR_METHODS:
func_name = STR_METHODS[method]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'{func_name} "${{{var_name}}}" {args_str} >/dev/null'.strip())
return True
return False
def generate_call_statement(self, expr: CallExpr) -> str:
callee = expr.callee
location = getattr(expr, 'location', None) or getattr(callee, 'location', None)
if isinstance(callee, MemberAccess) and isinstance(callee.object, Identifier):
if callee.object.name == "json" and callee.member == "stringify":
if len(expr.arguments) == 1 and isinstance(expr.arguments[0], Identifier):
dict_name = expr.arguments[0].name
if dict_name in self.dict_vars:
return f'__ct_json_stringify "{dict_name}"'
if callee.object.name == "json" and callee.member == "get":
args = [self._generate_call_arg(arg) for arg in expr.arguments]
return f'__ct_json_get "{args[0]}" "{args[1]}"'
args = [self._generate_call_arg(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
if isinstance(callee, Identifier):
return self._generate_builtin_call(callee.name, args, args_str)
if isinstance(callee, MemberAccess):
return self._generate_member_call(callee, args, args_str, location)
return ""
def _generate_builtin_call(self, name: str, args: list, args_str: str) -> str:
"""Generate builtin function call."""
if name == "print":
return f'__ct_print "{args[0]}"' if args else '__ct_print ""'
elif name == "range":
return f'__ct_range {args_str}'
elif name == "exit":
return f'__ct_exit {args_str}'
elif name == "len":
return f'__ct_str_len {args_str}'
elif name == "is_number":
return f'__ct_is_number {args_str}'
elif name == "is_empty":
return f'__ct_is_empty {args_str}'
elif name == "chr":
return f'__ct_str_chr {args_str}'
elif name == "ord":
return f'__ct_str_ord {args_str}'
elif name == "assert":
if len(args) >= 2:
return f'__ct_assert "{args[0]}" "{args[1]}"'
return f'__ct_assert "{args[0]}"'
elif name == "assert_eq":
return f'__ct_assert_eq {args_str}'
elif name == "pid":
return '__ct_pid'
else:
resolved = self._resolve_name(name)
if resolved != name:
return f'{resolved} {args_str}'
if self._is_callback_var(name):
return f'"${{{name}}}" {args_str}'
return f'{name} {args_str}'
def _is_callback_var(self, name: str) -> bool:
"""Check if name is a variable holding a callback (function name)."""
if name in getattr(self, 'callback_vars', set()):
return True
resolved = self._resolve_name(name)
if resolved in self.functions or resolved in self.classes:
return False
if name in self.local_vars:
return True
if name in getattr(self, 'current_param_positions', {}):
return True
return False
def _is_shell_command(self, expr) -> bool:
if self._is_shell_exec(expr):
return True
if isinstance(expr, CallExpr) and isinstance(expr.callee, Identifier):
name = expr.callee.name
resolved = self._resolve_name(name)
if (resolved not in self.functions and
name not in BUILTIN_FUNCS and
resolved not in self.classes and
not self._is_callback_var(name)):
return True
return False
def _extract_shell_command_str(self, expr) -> str:
if self._is_shell_exec(expr):
return self._extract_shell_command(expr)
if isinstance(expr, CallExpr) and isinstance(expr.callee, Identifier):
name = expr.callee.name
if expr.arguments:
args = [self.generate_expr(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
return f'{name} {args_str}'
return name
return ""
def _generate_member_call(self, callee: MemberAccess, args: list, args_str: str, location=None) -> str:
"""Generate member access call."""
obj = self.generate_expr(callee.object)
method = callee.member
if isinstance(callee.object, ThisExpr) and self.current_class:
key = (self.current_class, method)
if key in self.inlineable_methods and not args_str.strip():
return f'__CT_RET={self.inlineable_methods[key]}'
return f'__ct_class_{self.current_class}_{method} "$this" {args_str}'
if isinstance(callee.object, Identifier):
obj_name = callee.object.name
resolved = self._resolve_qualified(obj_name, method)
if resolved is not None:
return f'{resolved} {args_str}'
if obj_name in self.busing_names:
return f'{method} {args_str}'
result = self._generate_stdlib_call(obj_name, method, args, args_str, location)
if result:
return result
if isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, ThisExpr):
result = self._generate_this_field_call(callee, args_str, location)
if result:
return result
if isinstance(callee.object, MemberAccess) and isinstance(callee.object.object, Identifier):
result = self._generate_var_field_call(callee, args_str, location)
if result:
return result
var_name = callee.object.name if isinstance(callee.object, Identifier) else None
if var_name:
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(var_name, var_name)
if mapped_name in self.array_vars:
if method in ARR_METHODS:
arr_ref = mapped_name
if method == "push" and len(args) == 1:
return f'{mapped_name}+=("{args[0]}")'
return f'{ARR_METHODS[method]} "{arr_ref}" {args_str}'.strip()
else:
self._validate_type_method("array", method, location)
if mapped_name in self.dict_vars:
if method in DICT_METHODS:
dict_ref = mapped_name
return f'{DICT_METHODS_MAP[method]} "{dict_ref}" {args_str}'.strip()
else:
self._validate_type_method("dict", method, location)
if var_name and var_name in self.file_handle_vars:
if method in FILE_HANDLE_METHODS:
return f'{FILE_HANDLE_METHODS_MAP[method]} "${var_name}" {args_str}'.strip()
else:
self._validate_type_method("file_handle", method, location)
if var_name and var_name in self.process_handle_vars:
if method in PROCESS_HANDLE_METHODS:
cp = self.process_handle_map[var_name]
if method == "read":
return f'read -r {RET_VAR} <&${cp}_rd && echo "${RET_VAR}"'
elif method == "write":
return f'echo {args_str} >&${cp}_wr'
elif method == "close":
return f'exec {{{cp}_wr}}>&-'
elif method == "kill":
return f'kill ${cp}_pid 2>/dev/null || true'
elif method == "wait":
return f'wait ${cp}_pid 2>/dev/null || true'
else:
self._validate_type_method("process_handle", method, location)
if method in STR_METHODS:
return f'{STR_METHODS[method]} "{obj}" {args_str}'.strip()
return f'__ct_call_method "{obj}" "{method}" {args_str}'
def _generate_stdlib_call(self, obj_name: str, method: str, args: list, args_str: str, location=None) -> str:
"""Generate stdlib call. Returns None if not stdlib."""
if obj_name in BUILTIN_NAMESPACES:
self._validate_namespace_method(obj_name, method, location)
if obj_name == "http":
return f'__ct_http_{method} {args_str}'
elif obj_name == "fs":
return f'__ct_fs_{method} {args_str}'
elif obj_name == "json":
if method == "marshal":
return f'__ct_json_marshal {args_str}'
return f'__ct_json_{method} {args_str}'
elif obj_name == "reflect":
return f'__ct_reflect_{method} {args_str}'
elif obj_name == "logger" and method in ("info", "warn", "error", "debug"):
return f'__ct_logger_{method} {args_str}'
elif obj_name == "regex":
return f'__ct_regex_{method} {args_str}'
elif obj_name == "args":
if method == "count":
return '__ct_args_count'
elif method == "get":
return f'__ct_args_get {args_str}'
elif obj_name == "shell":
if method == "exec":
return args[0] if args else ""
elif method == "capture":
return f'$({args[0]})' if args else ""
elif method == "source":
return f'source "{args[0]}"' if args else ""
elif obj_name == "time":
if method == "now":
return '__ct_time_now'
elif method == "ms":
return '__ct_time_ms'
elif obj_name == "math":
return f'__ct_math_{method} {args_str}'
return None
def _generate_this_field_call(self, callee: MemberAccess, args_str: str, location=None) -> str:
"""Generate this.field.method() call."""
field_name = callee.object.member
method = callee.member
field_type = self.class_field_types.get((self.current_class, field_name)) if self.current_class else None
if field_type == "array":
if method in ARR_METHODS:
arr_name = f'"${{this}}_{field_name}"'
return f'{ARR_METHODS[method]} {arr_name} {args_str}'.strip()
else:
self._validate_type_method("array", method, location)
if field_type == "dict":
if method in DICT_METHODS:
dict_ref = f'"${{this}}_{field_name}"'
return f'{DICT_METHODS_MAP[method]} {dict_ref} {args_str}'.strip()
else:
self._validate_type_method("dict", method, location)
if field_type == "object":
obj_ref = f'${{__CT_OBJ["$this.{field_name}"]}}'
return f'__ct_call_method "{obj_ref}" "{method}" {args_str}'
return None
def _generate_var_field_call(self, callee: MemberAccess, args_str: str, location=None) -> str:
"""Generate var.field.method() call."""
var_name = callee.object.object.name
field_name = callee.object.member
method = callee.member
is_array_field = any(
self.class_field_types.get((cls, field_name)) == "array"
for cls in self.classes
)
is_dict_field = any(
self.class_field_types.get((cls, field_name)) == "dict"
for cls in self.classes
)
is_object_field = any(
self.class_field_types.get((cls, field_name)) == "object"
for cls in self.classes
)
if is_array_field:
if method in ARR_METHODS:
return f'{ARR_METHODS[method]} "${{{var_name}}}_{field_name}" {args_str}'.strip()
else:
self._validate_type_method("array", method, location)
if is_dict_field:
if method in DICT_METHODS:
return f'{DICT_METHODS_MAP[method]} "${{{var_name}}}_{field_name}" {args_str}'.strip()
else:
self._validate_type_method("dict", method, location)
if is_object_field:
obj_ref = f'${{__CT_OBJ["${{{var_name}}}.{field_name}"]}}'
return f'__ct_call_method "{obj_ref}" "{method}" {args_str}'
return None
bootstrap/errors.py deleted 100644 → 0
View file @ 5f7339cd
from dataclasses import dataclass
from typing import List, Optional
import sys
@dataclass
class CompileError:
message: str
filename: str
line: int
column: int
hint: Optional[str] = None
def __str__ (self):
result = f"Error: {self.message}\n --> {self.filename}:{self.line}:{self.column}"
if self.hint:
result += f"\n Hint: {self.hint}"
return result
class ErrorCollector:
def __init__ (self):
self.errors: List[CompileError] = []
self._seen: set = set()
def _error_key(self, msg: str, filename: str, line: int, column: int) -> tuple:
return (msg, filename, line, column)
def add (self, error: CompileError):
key = self._error_key(error.message, error.filename, error.line, error.column)
if key not in self._seen:
self._seen.add(key)
self.errors.append(error)
def add_error (self, message: str, filename: str, line: int, column: int, hint: str = None):
key = self._error_key(message, filename, line, column)
if key not in self._seen:
self._seen.add(key)
self.errors.append (CompileError (
message=message,
filename=filename,
line=line,
column=column,
hint=hint
))
def has_errors (self) -> bool:
return len (self.errors) > 0
def print_errors (self):
for error in self.errors:
print (str (error), file=sys.stderr)
print (file=sys.stderr)
def clear (self):
self.errors = []
bootstrap/expr_codegen.py deleted 100644 → 0
View file @ 5f7339cd
import re
from .ast_nodes import (
Expression, IntegerLiteral, FloatLiteral, StringLiteral, BoolLiteral,
NilLiteral, Identifier, ThisExpr, ArrayLiteral, DictLiteral, BinaryOp,
UnaryOp, CallExpr, MemberAccess, IndexAccess, Lambda, NewExpr, AsyncExpr,
BaseCall, Block, ReturnStmt
)
class ExprMixin:
"""Mixin for expression generation."""
def generate_expr(self, expr: Expression) -> str:
if isinstance(expr, IntegerLiteral):
return str(expr.value)
if isinstance(expr, FloatLiteral):
return str(expr.value)
if isinstance(expr, StringLiteral):
return self._generate_string_literal(expr)
if isinstance(expr, BoolLiteral):
return "true" if expr.value else "false"
if isinstance(expr, NilLiteral):
return ""
if isinstance(expr, Identifier):
name = expr.name
param_map = getattr(self, 'param_name_map', {})
if name in param_map:
name = param_map[name]
return f"${{{name}}}"
if isinstance(expr, ThisExpr):
return "$this"
if isinstance(expr, ArrayLiteral):
elements = [self.generate_expr(e) for e in expr.elements]
return "(" + " ".join([f'"{e}"' for e in elements]) + ")"
if isinstance(expr, DictLiteral):
pairs = []
for k, v in expr.pairs:
key = self.generate_expr(k)
val = self.generate_expr(v)
pairs.append(f'[{key}]="{val}"')
return "(" + " ".join(pairs) + ")"
if isinstance(expr, BinaryOp):
return self._generate_binary_op(expr)
if isinstance(expr, UnaryOp):
return self._generate_unary_op(expr)
if isinstance(expr, CallExpr):
return self._generate_call_expr(expr)
if isinstance(expr, MemberAccess):
return self._generate_member_access(expr)
if isinstance(expr, IndexAccess):
return self._generate_index_access(expr)
if isinstance(expr, Lambda):
return self.generate_lambda(expr)
if isinstance(expr, NewExpr):
resolved_class = self._resolve_name(expr.class_name)
args = [self.generate_expr(a) for a in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
return f'$({resolved_class} {args_str}; echo "$__ct_last_instance")'
if isinstance(expr, AsyncExpr):
return self._generate_async_expr(expr)
if isinstance(expr, BaseCall):
args = [self.generate_expr(a) for a in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
return f'# base({args_str})'
return ""
def _generate_string_literal(self, expr: StringLiteral) -> str:
"""Handle string interpolation."""
value = expr.value
value = value.replace('\\', '\\\\')
value = value.replace('"', '\\"')
value = value.replace('`', '\\`')
def escape_dollar(match):
pos = match.start()
rest = value[pos + 1:]
if not rest.startswith('{'):
return '\\$'
if rest.startswith('{{'):
return '\\$'
if rest.startswith('{__CT_') or rest.startswith('{['):
return '\\$'
brace_content = rest[1:]
close_pos = brace_content.find('}')
if close_pos == -1:
return '\\$'
return '$'
value = re.sub(r'\$', escape_dollar, value)
def replace_dollar_brace(match):
content = match.group(1)
if content.startswith('__CT_') or '[' in content:
return match.group(0)
if '.' in content:
parts = content.split('.', 1)
if parts[0] == 'this':
return f'\\$${{__CT_OBJ["$this.{parts[1]}"]}}'
ph_vars = getattr(self, 'process_handle_vars', set())
ph_map = getattr(self, 'process_handle_map', {})
if parts[0] in ph_vars and parts[1] == 'pid':
cp = ph_map[parts[0]]
return f'\\$${cp}_pid'
return f'\\$${{__CT_OBJ["${{{parts[0]}}}.{parts[1]}"]:-}}'
return f'\\$${{{content}}}'
def replace_interpolation(match):
content = match.group(1)
if '[' in content or content.startswith('__'):
return match.group(0)
if '(' in content:
return self._handle_interpolation_call(content)
comparison_result = self._handle_interpolation_operator(content)
if comparison_result:
return comparison_result
if '.' in content:
parts = content.split('.', 1)
if parts[0] == 'this':
return f'${{__CT_OBJ["$this.{parts[1]}"]}}'
ph_vars = getattr(self, 'process_handle_vars', set())
ph_map = getattr(self, 'process_handle_map', {})
if parts[0] in ph_vars and parts[1] == 'pid':
cp = ph_map[parts[0]]
return f'${cp}_pid'
return f'${{__CT_OBJ["${{{parts[0]}}}.{parts[1]}"]:-}}'
return f'${{{content}}}'
value = value.replace('\x00DOLLAR\x00{', '\x01ESCAPED_DOLLAR_BRACE\x01')
value = value.replace('\x00DOLLAR\x00', '\\$')
value = value.replace('\x00LBRACE\x00', '\x01ESCAPED_LBRACE\x01')
value = value.replace('\x00RBRACE\x00', '\x01ESCAPED_RBRACE\x01')
value = re.sub(r'\$\{(?!\{)([^}]+)\}', replace_dollar_brace, value)
value = re.sub(r'(?<!\$)\{([^}\[]+)\}', replace_interpolation, value)
value = value.replace('\x01ESCAPED_DOLLAR_BRACE\x01', '\\${')
value = value.replace('\x01ESCAPED_LBRACE\x01', '{')
value = value.replace('\x01ESCAPED_RBRACE\x01', '}')
return value
def _handle_interpolation_operator(self, content: str) -> str:
"""Handle operators in string interpolation like {a == b}."""
comparison_ops = [('==', '=='), ('!=', '!='), ('<=', '-le'), ('>=', '-ge'), ('<', '-lt'), ('>', '-gt')]
for ct_op, bash_op in comparison_ops:
if ct_op in content:
parts = content.split(ct_op, 1)
if len(parts) == 2:
left = parts[0].strip()
right = parts[1].strip()
left_bash = self._interpolation_operand_to_bash(left)
right_bash = self._interpolation_operand_to_bash(right)
if ct_op in ('==', '!='):
return f'$([[ "{left_bash}" {bash_op} "{right_bash}" ]] && echo true || echo false)'
else:
return f'$([[ {left_bash} {bash_op} {right_bash} ]] && echo true || echo false)'
arith_ops = ['+', '-', '*', '/', '%']
for op in arith_ops:
if op in content and not content.startswith(op):
parts = content.split(op, 1)
if len(parts) == 2 and parts[0].strip() and parts[1].strip():
left = parts[0].strip()
right = parts[1].strip()
left_bash = self._interpolation_operand_to_bash(left)
right_bash = self._interpolation_operand_to_bash(right)
return f'$(({left_bash} {op} {right_bash}))'
return None
def _interpolation_operand_to_bash(self, operand: str) -> str:
"""Convert an operand in string interpolation to bash syntax."""
operand = operand.strip()
if operand.startswith('"') and operand.endswith('"'):
return operand[1:-1]
if operand.isdigit() or (operand.startswith('-') and operand[1:].isdigit()):
return operand
if '.' in operand:
parts = operand.split('.', 1)
if parts[0] == 'this':
return f'${{__CT_OBJ["$this.{parts[1]}"]}}'
else:
return f'${{__CT_OBJ["${{{parts[0]}}}.{parts[1]}"]:-}}'
return f'${{{operand}}}'
def _handle_interpolation_call(self, content: str) -> str:
"""Handle function/method calls in string interpolation."""
if '.' in content and not content.startswith('('):
paren_idx = content.find('(')
before_paren = content[:paren_idx]
dots_count = before_paren.count('.')
if dots_count >= 2:
parts = before_paren.split('.')
obj = parts[0].strip()
field = parts[1].strip()
method = parts[2].strip()
args_csv = content[paren_idx + 1:-1]
args_list = self._parse_args_with_parens(args_csv) if args_csv else []
args_bash = ' '.join([self._convert_arg_to_bash(a) for a in args_list])
is_array_field = any(
getattr(self, 'class_field_types', {}).get((cls, field)) == "array"
for cls in getattr(self, 'classes', [])
)
is_dict_field = any(
getattr(self, 'class_field_types', {}).get((cls, field)) == "dict"
for cls in getattr(self, 'classes', [])
)
if is_array_field and method in ('len', 'get', 'shift', 'join', 'slice'):
return f'$(__ct_arr_{method} "${{{obj}}}_{field}" {args_bash})'.replace(' ', ' ').strip()
elif is_dict_field and method in ('get', 'has', 'keys', 'del'):
return f'$(__ct_dict_{method} "${{{obj}}}_{field}" {args_bash})'.replace(' ', ' ').strip()
dot_idx = content.find('.')
if dot_idx < paren_idx:
parts = content.split('.', 1)
obj = parts[0]
rest = parts[1]
method_paren_idx = rest.find('(')
method = rest[:method_paren_idx].strip()
args_csv = rest[method_paren_idx + 1:-1]
args_list = self._parse_args_with_parens(args_csv) if args_csv else []
args_bash = ' '.join([self._convert_arg_to_bash(a) for a in args_list])
is_var = (obj in getattr(self, 'array_vars', set()) or
obj in getattr(self, 'dict_vars', set()) or
obj in getattr(self, 'local_vars', set()) or
obj in getattr(self, 'global_vars', set()))
if obj == 'str' and not is_var:
return f'$(__ct_str_{method} {args_bash})'
elif obj == 'arr' and not is_var:
return f'$(__ct_arr_{method} {args_bash})'
elif obj == 'args' and not is_var:
if method == 'count':
return '$(__ct_args_count)'
elif method == 'get':
return f'$(__ct_args_get {args_bash})'
if obj in getattr(self, 'array_vars', set()) and method in ('len', 'get', 'shift', 'join', 'slice', 'push', 'pop', 'set'):
arr_ref = f'${{{obj}}}' if obj in getattr(self, 'nameref_vars', set()) else obj
return f'$(__ct_arr_{method} "{arr_ref}" {args_bash})'.replace(' ', ' ').strip()
if obj in getattr(self, 'dict_vars', set()) and method in ('get', 'has', 'keys', 'set', 'del'):
dict_ref = f'${{{obj}}}' if obj in getattr(self, 'nameref_vars', set()) else obj
return f'$(__ct_dict_{method} "{dict_ref}" {args_bash})'.replace(' ', ' ').strip()
str_methods = ('len', 'upper', 'lower', 'trim', 'contains', 'starts', 'ends', 'index', 'replace', 'substr', 'split', 'charAt')
if method in str_methods:
if method == 'charAt':
return f'${{$(__ct_str_char_at "${{{obj}}}" {args_bash})%X}}'.replace(' ', ' ').strip()
return f'$(__ct_str_{method} "${{{obj}}}" {args_bash})'.replace(' ', ' ').strip()
return f'$(__ct_call_method "${{{obj}}}" "{method}" {args_bash})'
paren_idx = content.find('(')
func_name = content[:paren_idx].strip()
args_part = content[paren_idx + 1:-1].strip()
interp_builtin_map = {
'print': '__ct_print', 'len': '__ct_str_len', 'exit': '__ct_exit',
'range': '__ct_range', 'is_number': '__ct_is_number',
'is_empty': '__ct_is_empty', 'pid': '__ct_pid',
'random': '__ct_random', 'random_range': '__ct_random_range',
}
bash_name = interp_builtin_map.get(func_name, func_name)
if args_part:
args_list = self._parse_args_with_parens(args_part)
args_bash = ' '.join([self._convert_arg_to_bash(a) for a in args_list])
return f'$({bash_name} {args_bash})'
else:
return f'$({bash_name})'
def _parse_args_with_parens(self, args_str: str) -> list:
"""Parse comma-separated args, respecting nested parentheses."""
args = []
current = ""
depth = 0
for ch in args_str:
if ch == '(':
depth += 1
current += ch
elif ch == ')':
depth -= 1
current += ch
elif ch == ',' and depth == 0:
args.append(current.strip())
current = ""
else:
current += ch
if current.strip():
args.append(current.strip())
return args
def _convert_arg_to_bash(self, arg: str) -> str:
"""Convert a single argument to bash syntax."""
arg = arg.strip()
if not arg:
return '""'
if '(' in arg and arg.endswith(')'):
paren_idx = arg.find('(')
func_name = arg[:paren_idx].strip()
inner_args = arg[paren_idx + 1:-1].strip()
if inner_args:
inner_list = self._parse_args_with_parens(inner_args)
inner_bash = ' '.join([self._convert_arg_to_bash(a) for a in inner_list])
return f'$({func_name} {inner_bash})'
else:
return f'$({func_name})'
if arg.isdigit() or (arg.startswith('-') and arg[1:].isdigit()):
return f'"{arg}"'
if arg.startswith('"') and arg.endswith('"'):
return arg
return f'"${{{arg}}}"'
def _generate_binary_op(self, expr: BinaryOp) -> str:
left = self.generate_expr(expr.left)
right = self.generate_expr(expr.right)
op = expr.operator
if op == "+":
return f"$(({left} + {right}))"
elif op == "..":
return f"{left}{right}"
elif op in ("-", "*", "/", "%"):
return f"$(({left} {op} {right}))"
elif op == "==":
return f'[[ "{left}" == "{right}" ]]'
elif op == "!=":
return f'[[ "{left}" != "{right}" ]]'
elif op == "<":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ "{left}" < "{right}" ]]'
return f'[[ {left} -lt {right} ]]'
elif op == ">":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ "{left}" > "{right}" ]]'
return f'[[ {left} -gt {right} ]]'
elif op == "<=":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ ! "{left}" > "{right}" ]]'
return f'[[ {left} -le {right} ]]'
elif op == ">=":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ ! "{left}" < "{right}" ]]'
return f'[[ {left} -ge {right} ]]'
elif op == "&&":
return f'{left} && {right}'
elif op == "||":
return f'{left} || {right}'
elif op == "|":
return self._generate_pipe_expr(expr)
return f"$(({left} {op} {right}))"
def _generate_unary_op(self, expr: UnaryOp) -> str:
operand = self.generate_expr(expr.operand)
if expr.operator == "!":
return f'! {operand}'
elif expr.operator == "-":
return f'$((-{operand}))'
return operand
def _generate_call_expr(self, expr: CallExpr) -> str:
if isinstance(expr.callee, Identifier) and expr.callee.name == "len":
if len(expr.arguments) == 1:
arg = expr.arguments[0]
if isinstance(arg, MemberAccess) and isinstance(arg.object, ThisExpr):
return f'${{#__CT_OBJ["$this.{arg.member}"]}}'
if isinstance(expr.callee, MemberAccess):
if isinstance(expr.callee.object, Identifier) and expr.callee.object.name == "str":
result = self._inline_str_method(expr)
if result:
return result
if isinstance(expr.callee.object, Identifier) and expr.callee.object.name == "arr":
result = self._inline_arr_method(expr)
if result:
return result
if isinstance(expr.callee.object, ThisExpr) and self.current_class:
method = expr.callee.member
key = (self.current_class, method)
if key in self.inlineable_methods and len(expr.arguments) == 0:
return self.inlineable_methods[key]
if isinstance(expr.callee.object, Identifier) and expr.callee.object.name == "shell":
if expr.callee.member == "capture":
return self.generate_call_statement(expr)
return f"$({self.generate_call_statement(expr)})"
def _inline_str_method(self, expr: CallExpr) -> str:
"""Inline common str methods."""
method = expr.callee.member
if method == "len" and len(expr.arguments) == 1:
arg = expr.arguments[0]
if isinstance(arg, MemberAccess) and isinstance(arg.object, ThisExpr):
return f'${{#__CT_OBJ["$this.{arg.member}"]}}'
if isinstance(arg, Identifier):
return f'${{#{arg.name}}}'
if method == "charAt" and len(expr.arguments) == 2:
arg0, arg1 = expr.arguments
if isinstance(arg0, MemberAccess) and isinstance(arg0.object, ThisExpr):
pos = self.generate_expr(arg1)
return f'${{__CT_OBJ["$this.{arg0.member}"]:{pos}:1}}'
str_val = self.generate_expr(arg0)
if str_val.startswith('${') and str_val.endswith('}'):
str_val = str_val[2:-1]
elif str_val.startswith('$') and not str_val.startswith('$('):
str_val = str_val[1:]
pos = self.generate_expr(arg1)
return f'${{{str_val}:{pos}:1}}'
return None
def _inline_arr_method(self, expr: CallExpr) -> str:
"""Inline common arr methods."""
method = expr.callee.member
if method == "get" and len(expr.arguments) == 2:
arr_arg = expr.arguments[0]
idx_arg = expr.arguments[1]
if isinstance(arr_arg, Identifier):
arr_name = arr_arg.name
idx = self.generate_expr(idx_arg)
return f'${{{arr_name}[{idx}]}}'
if method == "len" and len(expr.arguments) == 1:
arr_arg = expr.arguments[0]
if isinstance(arr_arg, Identifier):
return f'${{#{arr_arg.name}[@]}}'
return None
def _generate_async_expr(self, expr: AsyncExpr) -> str:
from .constants import COPROC_PREFIX
self.coproc_counter += 1
cp_name = f'{COPROC_PREFIX}{self.coproc_counter}'
inner = expr.expression
if isinstance(inner, CallExpr) and self._is_shell_command(inner):
cmd_str = self._extract_shell_command_str(inner)
elif isinstance(inner, CallExpr) and isinstance(inner.callee, Identifier):
func_name = inner.callee.name
args_str = self._generate_call_args_str(inner.arguments)
cmd_str = f'exec 3>&1; {func_name} {args_str}'.strip()
else:
cmd_str = self.generate_expr(inner)
self.emit(f'coproc {cp_name} {{ {cmd_str}; }}')
self.emit(f'exec {{{cp_name}_wr}}>&${{{cp_name}[1]}}')
self.emit(f'exec {{{cp_name}_rd}}<&${{{cp_name}[0]}}')
self.emit(f'{cp_name}_pid=${cp_name}_PID')
self.emit(f'eval "exec ${{{cp_name}[1]}}>&-"')
return ""
def _generate_member_access(self, expr: MemberAccess) -> str:
if isinstance(expr.object, Identifier) and expr.object.name == "env":
return f'${{{expr.member}}}'
if isinstance(expr.object, Identifier):
obj_name = expr.object.name
if obj_name in getattr(self, 'process_handle_vars', set()) and expr.member == "pid":
cp = self.process_handle_map[obj_name]
return f'${cp}_pid'
if isinstance(expr.object, Identifier):
obj_name = expr.object.name
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(obj_name, obj_name)
if mapped_name in getattr(self, 'dict_vars', set()):
return f'${{{mapped_name}["{expr.member}"]}}'
field_name = expr.member
is_array_field = any(
self.class_field_types.get((cls, field_name)) == "array"
for cls in self.classes
)
is_dict_field = any(
self.class_field_types.get((cls, field_name)) == "dict"
for cls in self.classes
)
if isinstance(expr.object, ThisExpr):
if is_array_field or is_dict_field:
return f'${{this}}_{field_name}'
return f'${{__CT_OBJ["$this.{expr.member}"]}}'
obj = self.generate_expr(expr.object)
if obj.startswith('${') and obj.endswith('}'):
var_name = obj[2:-1]
if is_array_field or is_dict_field:
return f'${{{var_name}}}_{field_name}'
return f'${{__CT_OBJ["${{{var_name}}}.{expr.member}"]:-}}'
if is_array_field or is_dict_field:
return f'{obj}_{field_name}'
return f'${{__CT_OBJ["{obj}.{expr.member}"]:-}}'
def _generate_index_access(self, expr: IndexAccess) -> str:
obj = self.generate_expr(expr.object)
idx = self.generate_expr(expr.index)
if obj.startswith('${') and obj.endswith('}'):
obj = obj[2:-1]
elif obj.startswith('$'):
obj = obj[1:]
return f'${{{obj}[{idx}]}}'
def is_string_comparison(self, left_expr: Expression, right_expr: Expression) -> bool:
"""Check if this is a string comparison (vs numeric)."""
if isinstance(left_expr, StringLiteral) and len(left_expr.value) == 1:
return True
if isinstance(right_expr, StringLiteral) and len(right_expr.value) == 1:
return True
if isinstance(left_expr, StringLiteral):
try:
float(left_expr.value)
except ValueError:
return True
if isinstance(right_expr, StringLiteral):
try:
float(right_expr.value)
except ValueError:
return True
return False
def generate_condition(self, expr: Expression) -> str:
if isinstance(expr, BinaryOp):
return self._generate_binary_condition(expr)
if isinstance(expr, UnaryOp) and expr.operator == "!":
inner = self.generate_condition(expr.operand)
return f'! {inner}'
if isinstance(expr, Identifier):
return f'[[ "${expr.name}" == "true" ]]'
if isinstance(expr, BoolLiteral):
return "true" if expr.value else "false"
if isinstance(expr, CallExpr):
return self._generate_call_condition(expr)
result = self.generate_expr(expr)
return f'[[ -n "{result}" ]]'
def _generate_binary_condition(self, expr: BinaryOp) -> str:
left = self.generate_expr(expr.left)
right = self.generate_expr(expr.right)
op = expr.operator
if op == "==":
return f'[[ "{left}" == "{right}" ]]'
elif op == "!=":
return f'[[ "{left}" != "{right}" ]]'
elif op == "<":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ "{left}" < "{right}" ]]'
return f'[[ {left} -lt {right} ]]'
elif op == ">":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ "{left}" > "{right}" ]]'
return f'[[ {left} -gt {right} ]]'
elif op == "<=":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ ! "{left}" > "{right}" ]]'
return f'[[ {left} -le {right} ]]'
elif op == ">=":
if self.is_string_comparison(expr.left, expr.right):
return f'[[ ! "{left}" < "{right}" ]]'
return f'[[ {left} -ge {right} ]]'
elif op == "&&":
l = self.generate_condition(expr.left)
r = self.generate_condition(expr.right)
return f'{{ {l} && {r}; }}'
elif op == "||":
l = self.generate_condition(expr.left)
r = self.generate_condition(expr.right)
return f'{{ {l} || {r}; }}'
return f'[[ "{left}" == "{right}" ]]'
def _generate_call_condition(self, expr: CallExpr) -> str:
"""Generate condition for function calls."""
if isinstance(expr.callee, MemberAccess):
if isinstance(expr.callee.object, Identifier):
obj_name = expr.callee.object.name
method = expr.callee.member
if obj_name == "str":
if method == "contains" and len(expr.arguments) == 2:
haystack = self.generate_expr(expr.arguments[0])
needle = self.generate_expr(expr.arguments[1])
return f'[[ "{haystack}" == *"{needle}"* ]]'
elif method == "starts" and len(expr.arguments) == 2:
s = self.generate_expr(expr.arguments[0])
prefix = self.generate_expr(expr.arguments[1])
return f'[[ "{s}" == "{prefix}"* ]]'
elif method == "ends" and len(expr.arguments) == 2:
s = self.generate_expr(expr.arguments[0])
suffix = self.generate_expr(expr.arguments[1])
return f'[[ "{s}" == *"{suffix}" ]]'
if obj_name == "fs" and method == "exists" and len(expr.arguments) == 1:
path = self.generate_expr(expr.arguments[0])
return f'[[ -e "{path}" ]]'
if obj_name == "regex" and method == "match" and len(expr.arguments) == 2:
s = self.generate_expr(expr.arguments[0])
pattern = self.generate_expr(expr.arguments[1])
return f'[[ "{s}" =~ {pattern} ]]'
if isinstance(expr.callee, Identifier):
if expr.callee.name == "is_number" and len(expr.arguments) == 1:
val = self.generate_expr(expr.arguments[0])
return f'[[ "{val}" =~ ^-?[0-9]+$ ]]'
elif expr.callee.name == "is_empty" and len(expr.arguments) == 1:
val = self.generate_expr(expr.arguments[0])
return f'[[ -z "{val}" ]]'
result = self.generate_expr(expr)
return f'[[ "{result}" == "true" ]]'
def generate_lambda(self, expr: Lambda) -> str:
"""Generate lambda and return its name."""
name = self.new_lambda_name()
self.generate_lambda_as_function(expr, name)
return name
def generate_lambda_as_function(self, expr: Lambda, name: str):
"""Generate lambda as a named function."""
self.emit(f"{name} () {{")
with self.indented():
for i, param in enumerate(expr.params):
self.emit(f'local {param}="${{{i + 1}}}"')
if isinstance(expr.body, Block):
for stmt in expr.body.statements:
self.generate_statement(stmt)
else:
if self._is_boolean_expr(expr.body):
cond = self.generate_condition(expr.body)
self.emit(f'{cond} && echo "true" || echo "false"')
else:
result = self.generate_expr(expr.body)
self.emit(f'echo "{result}"')
self.emit("}")
self.emit()
def _is_boolean_expr(self, expr: Expression) -> bool:
"""Check if expression returns a boolean value."""
if isinstance(expr, BinaryOp):
return expr.operator in ("==", "!=", "<", ">", "<=", ">=", "&&", "||")
if isinstance(expr, UnaryOp):
return expr.operator == "!"
if isinstance(expr, BoolLiteral):
return True
return False
def generate_lvalue(self, expr: Expression) -> str:
"""Generate left-hand side of assignment (without $ prefix)."""
if isinstance(expr, Identifier):
return expr.name
elif isinstance(expr, MemberAccess):
if isinstance(expr.object, ThisExpr):
return f'"${{this}}_{expr.member}"'
obj = self.generate_lvalue(expr.object)
return f'{obj}_{expr.member}'
elif isinstance(expr, IndexAccess):
obj = self.generate_lvalue(expr.object)
idx = self.generate_expr(expr.index)
return f'{obj}[{idx}]'
return self.generate_expr(expr)
def _generate_pipe_expr(self, expr: BinaryOp) -> str:
"""Generate pipe expression as inline expression."""
elements = self._collect_pipe_chain(expr)
if all(self._is_shell_command(e) for e in elements):
commands = [self._extract_shell_command_str(e) for e in elements]
return f'$({" | ".join(commands)})'
if any(self._is_shell_command(e) for e in elements[1:]):
return self._generate_mixed_pipe_inline(elements)
return self._generate_functional_pipe_inline(elements)
def _generate_mixed_pipe_inline(self, elements: list) -> str:
"""Generate native bash pipe mixing CT functions and shell commands."""
parts = []
for elem in elements:
if self._is_shell_command(elem):
parts.append(self._extract_shell_command_str(elem))
elif isinstance(elem, CallExpr):
call_code = self.generate_call_statement(elem)
parts.append(f'{call_code} 3>&1')
elif isinstance(elem, Identifier):
parts.append(f'{elem.name} 3>&1')
else:
code = self.generate_expr(elem)
parts.append(code)
return f'$({" | ".join(parts)})'
def _collect_pipe_chain(self, expr: Expression) -> list:
"""Collect all elements in a pipe chain from left to right."""
if isinstance(expr, BinaryOp) and expr.operator == "|":
return self._collect_pipe_chain(expr.left) + self._collect_pipe_chain(expr.right)
return [expr]
def _is_shell_exec(self, expr: Expression) -> bool:
"""Check if expression is shell.exec() or shell.capture()."""
if isinstance(expr, CallExpr) and isinstance(expr.callee, MemberAccess):
if isinstance(expr.callee.object, Identifier):
return (expr.callee.object.name == "shell" and
expr.callee.member in ("exec", "capture"))
return False
def _extract_shell_command(self, expr: Expression) -> str:
"""Extract shell command string from shell.exec()/shell.capture() call."""
if isinstance(expr, CallExpr) and expr.arguments:
arg = expr.arguments[0]
if isinstance(arg, StringLiteral):
return arg.value
return self.generate_expr(arg)
return ""
def _generate_functional_pipe_inline(self, elements: list) -> str:
"""Generate functional pipe as nested function calls."""
if len(elements) == 0:
return ""
if len(elements) == 1:
return self.generate_expr(elements[0])
result = self.generate_expr(elements[0])
for elem in elements[1:]:
if isinstance(elem, CallExpr):
result = self._wrap_pipe_call(elem, result)
elif isinstance(elem, Identifier):
result = f'$({elem.name} "{result}")'
else:
func = self.generate_expr(elem)
result = f'$({func} "{result}")'
return result
def _wrap_pipe_call(self, call_expr: CallExpr, prev_result: str) -> str:
"""Wrap a function call to receive piped result as first argument."""
if isinstance(call_expr.callee, Identifier):
func_name = call_expr.callee.name
args = [self.generate_expr(a) for a in call_expr.arguments]
all_args = [f'"{prev_result}"'] + [f'"{a}"' for a in args]
return f'$({func_name} {" ".join(all_args)})'
elif isinstance(call_expr.callee, MemberAccess):
call_code = self.generate_call_statement(call_expr)
if self._is_shell_command(call_expr):
cmd = self._extract_shell_command_str(call_expr)
return f'$(echo "{prev_result}" | {cmd})'
return f'$({call_code} "{prev_result}")'
return prev_result
bootstrap/lexer.py deleted 100644 → 0
View file @ 5f7339cd
from typing import List, Optional
from .tokens import Token, TokenType, KEYWORDS
from .errors import CompileError
class Lexer:
def __init__ (self, source: str, filename: str = "<stdin>"):
self.source = source
self.filename = filename
self.pos = 0
self.line = 1
self.column = 1
self.tokens: List[Token] = []
self.errors: List[CompileError] = []
def current (self) -> Optional[str]:
if self.pos >= len (self.source):
return None
return self.source[self.pos]
def peek (self, offset: int = 1) -> Optional[str]:
pos = self.pos + offset
if pos >= len (self.source):
return None
return self.source[pos]
def advance (self) -> Optional[str]:
ch = self.current ()
if ch is None:
return None
self.pos += 1
if ch == '\n':
self.line += 1
self.column = 1
else:
self.column += 1
return ch
def skip_whitespace (self):
while self.current () in (' ', '\t', '\r'):
self.advance ()
def add_token (self, type: TokenType, value=None, line=None, column=None):
self.tokens.append (Token (
type=type,
value=value,
line=line or self.line,
column=column or self.column
))
def error (self, message: str):
self.errors.append (CompileError (
message=message,
filename=self.filename,
line=self.line,
column=self.column
))
def read_string (self) -> str:
start_line = self.line
start_col = self.column
quote = self.advance ()
result = []
while True:
ch = self.current ()
if ch is None:
self.error ("Unterminated string")
break
if ch == quote:
self.advance ()
break
if ch == '\\':
self.advance ()
escaped = self.current ()
if escaped is None:
self.error ("Unterminated escape sequence")
break
escape_map = {
'n': '\n',
't': '\t',
'r': '\r',
'\\': '\\',
'{': '\x00LBRACE\x00',
'}': '\x00RBRACE\x00',
'$': '\x00DOLLAR\x00',
'"': '"',
"'": "'",
}
result.append (escape_map.get (escaped, escaped))
self.advance ()
else:
result.append (ch)
self.advance ()
return ''.join (result)
def read_number (self) -> Token:
start_col = self.column
result = []
is_float = False
while True:
ch = self.current ()
if ch is None:
break
if ch.isdigit ():
result.append (ch)
self.advance ()
elif ch == '.' and not is_float:
if self.peek () and self.peek ().isdigit ():
is_float = True
result.append (ch)
self.advance ()
else:
break
else:
break
value = ''.join (result)
if is_float:
return Token (TokenType.FLOAT, float (value), self.line, start_col)
return Token (TokenType.INTEGER, int (value), self.line, start_col)
def read_identifier (self) -> str:
result = []
while True:
ch = self.current ()
if ch is None:
break
if ch.isalnum () or ch == '_':
result.append (ch)
self.advance ()
elif ch == '-' and self.peek () and (self.peek ().isalpha () or self.peek () == '_'):
result.append (ch)
self.advance ()
else:
break
return ''.join (result)
def tokenize (self) -> List[Token]:
while True:
self.skip_whitespace ()
ch = self.current ()
if ch is None:
self.add_token (TokenType.EOF)
break
start_line = self.line
start_col = self.column
if ch == '#':
while self.current () and self.current () != '\n':
self.advance ()
continue
if ch == '\n':
self.add_token (TokenType.NEWLINE, line=start_line, column=start_col)
self.advance ()
continue
if ch in ('"', "'"):
value = self.read_string ()
self.add_token (TokenType.STRING, value, start_line, start_col)
continue
if ch.isdigit ():
token = self.read_number ()
self.tokens.append (token)
continue
if ch.isalpha () or ch == '_':
value = self.read_identifier ()
token_type = KEYWORDS.get (value, TokenType.IDENTIFIER)
self.add_token (token_type, value, start_line, start_col)
continue
if ch == '=' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.EQ, '==', start_line, start_col)
continue
if ch == '!' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.NEQ, '!=', start_line, start_col)
continue
if ch == '<' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.LTE, '<=', start_line, start_col)
continue
if ch == '>' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.GTE, '>=', start_line, start_col)
continue
if ch == '&' and self.peek () == '&':
self.advance ()
self.advance ()
self.add_token (TokenType.AND, '&&', start_line, start_col)
continue
if ch == '|' and self.peek () == '|':
self.advance ()
self.advance ()
self.add_token (TokenType.OR, '||', start_line, start_col)
continue
if ch == '=' and self.peek () == '>':
self.advance ()
self.advance ()
self.add_token (TokenType.ARROW, '=>', start_line, start_col)
continue
if ch == '+' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.PLUS_ASSIGN, '+=', start_line, start_col)
continue
if ch == '-' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.MINUS_ASSIGN, '-=', start_line, start_col)
continue
if ch == '*' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.STAR_ASSIGN, '*=', start_line, start_col)
continue
if ch == '/' and self.peek () == '=':
self.advance ()
self.advance ()
self.add_token (TokenType.SLASH_ASSIGN, '/=', start_line, start_col)
continue
if ch == '.' and self.peek () == '.' and self.peek (2) == '.':
self.advance ()
self.advance ()
self.advance ()
self.add_token (TokenType.DOTDOTDOT, '...', start_line, start_col)
continue
if ch == '.' and self.peek () == '.':
self.advance ()
self.advance ()
self.add_token (TokenType.DOTDOT, '..', start_line, start_col)
continue
single_char_tokens = {
'+': TokenType.PLUS,
'-': TokenType.MINUS,
'*': TokenType.STAR,
'/': TokenType.SLASH,
'%': TokenType.PERCENT,
'=': TokenType.ASSIGN,
'<': TokenType.LT,
'>': TokenType.GT,
'!': TokenType.NOT,
'.': TokenType.DOT,
':': TokenType.COLON,
'(': TokenType.LPAREN,
')': TokenType.RPAREN,
'{': TokenType.LBRACE,
'}': TokenType.RBRACE,
'[': TokenType.LBRACKET,
']': TokenType.RBRACKET,
',': TokenType.COMMA,
'@': TokenType.AT,
'|': TokenType.PIPE,
}
if ch in single_char_tokens:
self.advance ()
self.add_token (single_char_tokens[ch], ch, start_line, start_col)
continue
if ch == ';':
self.advance ()
continue
self.error (f"Unexpected character: '{ch}'")
self.advance ()
return self.tokens
bootstrap/meson.build deleted 100644 → 0
View file @ 5f7339cd
conf = configuration_data()
conf.set('PYTHON', py.full_path())
conf.set('pkgdatadir', pkgdatadir)
configure_file(
input: '../content.in',
output: 'content',
configuration: conf,
install: true,
install_dir: get_option('bindir'),
install_mode: 'r-xr-xr-x',
)
bootstrap_sources = [
'__init__.py',
'ast_nodes.py',
'awk_codegen.py',
'class_codegen.py',
'codegen.py',
'constants.py',
'cse_codegen.py',
'dce.py',
'decorator_codegen.py',
'dispatch_codegen.py',
'errors.py',
'expr_codegen.py',
'lexer.py',
'main.py',
'parser.py',
'stdlib.py',
'stmt_codegen.py',
'tokens.py',
]
install_data(bootstrap_sources,
install_dir: pkgdatadir / 'bootstrap')
methods_sources = [
'methods/__init__.py',
'methods/args.py',
'methods/array.py',
'methods/base.py',
'methods/core.py',
'methods/dict.py',
'methods/file_handle.py',
'methods/fs.py',
'methods/http.py',
'methods/json.py',
'methods/logger.py',
'methods/math.py',
'methods/process_handle.py',
'methods/reflect.py',
'methods/regex.py',
'methods/string.py',
'methods/time.py',
]
install_data(methods_sources,
install_dir: pkgdatadir / 'bootstrap' / 'methods')
bootstrap/parser.py deleted 100644 → 0
View file @ 5f7339cd
from typing import List, Optional, Callable, Union
from .tokens import Token, TokenType
from .ast_nodes import (
SourceLocation, Program, Declaration, Statement, Decorator, FunctionDecl,
Parameter, ClassDecl, ClassField, ConstructorDecl, Block, ReturnStmt,
BreakStmt, ContinueStmt, IfStmt, WhileStmt, ForStmt, ForeachStmt, WithStmt,
TryStmt, ThrowStmt, DeferStmt, AwaitStmt, OnSignalStmt, WhenStmt, WhenBranch, RangePattern,
ExpressionStmt, Assignment, IntegerLiteral, FloatLiteral, StringLiteral,
BoolLiteral, NilLiteral, ThisExpr, ArrayLiteral, DictLiteral, Identifier,
BinaryOp, UnaryOp, CallExpr, MemberAccess, IndexAccess, NewExpr, AsyncExpr, Lambda,
BaseCall, Expression, TypeAnnotation,
NamespaceDecl, UsingStmt, BusingStmt
)
from .errors import CompileError, ErrorCollector
class Parser:
def __init__ (self, tokens: List[Token], filename: str = "<stdin>"):
self.tokens = tokens
self.filename = filename
self.pos = 0
self.errors = ErrorCollector ()
def current (self) -> Token:
if self.pos >= len (self.tokens):
return self.tokens[-1]
return self.tokens[self.pos]
def peek (self, offset: int = 1) -> Token:
pos = self.pos + offset
if pos >= len (self.tokens):
return self.tokens[-1]
return self.tokens[pos]
def check (self, *types: TokenType) -> bool:
return self.current ().type in types
def match (self, *types: TokenType) -> bool:
if self.check (*types):
self.advance ()
return True
return False
def advance (self) -> Token:
token = self.current ()
if token.type != TokenType.EOF:
self.pos += 1
return token
def expect (self, type: TokenType, message: str = None) -> Token:
if self.check (type):
return self.advance ()
msg = message or f"Expected {type.name}, got {self.current ().type.name}"
self.error (msg)
return self.current ()
KEYWORD_AS_IDENT = {TokenType.ON, TokenType.ASYNC, TokenType.AWAIT}
def expect_name (self, message: str = "Expected identifier") -> Token:
if self.check (TokenType.IDENTIFIER):
return self.advance ()
if self.current ().type in self.KEYWORD_AS_IDENT:
tok = self.advance ()
tok.value = tok.value or tok.type.name.lower ()
return tok
self.error (message)
return self.current ()
def error (self, message: str, token: Token = None):
token = token or self.current ()
self.errors.add_error (
message=message,
filename=self.filename,
line=token.line,
column=token.column
)
def location (self, token: Token = None) -> SourceLocation:
token = token or self.current ()
return SourceLocation (token.line, token.column, self.filename)
def skip_newlines (self):
while self.match (TokenType.NEWLINE):
pass
def parse_type_annotation (self) -> Optional[TypeAnnotation]:
if not self.match (TokenType.COLON):
return None
return self.parse_type ()
def parse_type (self) -> TypeAnnotation:
loc = self.location ()
if self.check (TokenType.LPAREN):
return self.parse_function_type (loc)
name = self.expect (TokenType.IDENTIFIER, "Expected type name").value
if self.match (TokenType.LBRACKET):
if name == "dict":
key_type = self.parse_type ()
self.expect (TokenType.COMMA, "Expected ',' in dict type")
value_type = self.parse_type ()
self.expect (TokenType.RBRACKET, "Expected ']'")
return TypeAnnotation (name="dict", key_type=key_type, value_type=value_type, location=loc)
else:
self.expect (TokenType.RBRACKET, "Expected ']'")
return TypeAnnotation (name=name, is_array=True, location=loc)
return TypeAnnotation (name=name, location=loc)
def parse_function_type (self, loc: SourceLocation) -> TypeAnnotation:
self.expect (TokenType.LPAREN, "Expected '('")
param_types = []
if not self.check (TokenType.RPAREN):
while True:
param_types.append (self.parse_type ())
if not self.match (TokenType.COMMA):
break
self.expect (TokenType.RPAREN, "Expected ')'")
self.expect (TokenType.ARROW, "Expected '=>'")
return_type = self.parse_type ()
return TypeAnnotation (name="func", param_types=param_types, return_type=return_type, location=loc)
def parse (self) -> Program:
statements = []
self.skip_newlines ()
while not self.check (TokenType.EOF):
stmt = self.parse_declaration ()
if stmt:
statements.append (stmt)
self.skip_newlines ()
return Program (statements=statements, location=SourceLocation (1, 1, self.filename))
def parse_declaration (self) -> Optional[Union[Declaration, Statement]]:
decorators = []
while self.check (TokenType.AT):
decorators.append (self.parse_decorator ())
self.skip_newlines ()
if self.check (TokenType.FUNC):
return self.parse_function (decorators)
if self.check (TokenType.CLASS):
return self.parse_class ()
if self.check (TokenType.NAMESPACE):
return self.parse_namespace ()
if self.check (TokenType.USING):
return self.parse_using ()
if self.check (TokenType.BUSING):
return self.parse_busing ()
if decorators:
self.error ("Decorators can only be applied to functions or methods")
return self.parse_statement ()
def parse_decorator (self) -> Decorator:
loc = self.location ()
self.expect (TokenType.AT)
name = self.expect (TokenType.IDENTIFIER, "Expected decorator name").value
obj = None
if self.match (TokenType.DOT):
obj = name
name = self.expect_name ("Expected method name after '.'").value
arguments = []
if self.match (TokenType.LPAREN):
if not self.check (TokenType.RPAREN):
arguments = self.parse_decorator_args ()
self.expect (TokenType.RPAREN, "Expected ')' after decorator arguments")
return Decorator (name=name, arguments=arguments, object=obj, location=loc)
def parse_decorator_args (self) -> List[tuple]:
args = []
while True:
if self.check (TokenType.IDENTIFIER):
next_tok = self.peek ()
if next_tok.type == TokenType.ASSIGN:
name = self.advance ().value
self.advance ()
value = self.parse_expression ()
args.append ((name, value))
elif next_tok.type == TokenType.COLON:
name = self.advance ().value
self.advance ()
value = self.parse_expression ()
args.append ((name, value))
else:
value = self.parse_expression ()
args.append ((None, value))
else:
value = self.parse_expression ()
args.append ((None, value))
if not self.match (TokenType.COMMA):
break
return args
def parse_function (self, decorators: List[Decorator] = None) -> FunctionDecl:
loc = self.location ()
self.expect (TokenType.FUNC)
name = self.expect_name ("Expected function name").value
self.expect (TokenType.LPAREN, "Expected '(' after function name")
params = self.parse_parameters ()
self.expect (TokenType.RPAREN, "Expected ')' after parameters")
return_type = None
if self.check (TokenType.COLON):
return_type = self.parse_type_annotation ()
self.skip_newlines ()
body = self.parse_block ()
return FunctionDecl (
name=name,
params=params,
return_type=return_type,
body=body,
decorators=decorators or [],
location=loc
)
def parse_parameters (self) -> List[Parameter]:
params = []
if self.check (TokenType.RPAREN):
return params
while True:
name = self.expect (TokenType.IDENTIFIER, "Expected parameter name").value
type_annotation = None
is_variadic = False
default = None
if self.check (TokenType.COLON):
type_annotation = self.parse_type_annotation ()
if self.match (TokenType.DOTDOTDOT):
is_variadic = True
elif self.match (TokenType.ASSIGN):
default = self.parse_expression ()
params.append (Parameter (name=name, type_annotation=type_annotation, default=default, is_variadic=is_variadic))
if not self.match (TokenType.COMMA):
break
return params
def parse_class (self) -> ClassDecl:
loc = self.location ()
self.expect (TokenType.CLASS)
name = self.expect (TokenType.IDENTIFIER, "Expected class name").value
parent = None
if self.match (TokenType.COLON):
parent = self.expect (TokenType.IDENTIFIER, "Expected parent class name").value
self.skip_newlines ()
self.expect (TokenType.LBRACE, "Expected '{' after class declaration")
self.skip_newlines ()
fields = []
constructor = None
methods = []
while not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
decorators = []
while self.check (TokenType.AT):
decorators.append (self.parse_decorator ())
self.skip_newlines ()
if self.check (TokenType.CONSTRUCT):
constructor = self.parse_constructor ()
elif self.check (TokenType.FUNC):
methods.append (self.parse_function (decorators))
elif self.check (TokenType.IDENTIFIER):
field_loc = self.location ()
field_name = self.advance ().value
type_annotation = None
default_value = None
if self.check (TokenType.COLON):
type_annotation = self.parse_type_annotation ()
if self.match (TokenType.ASSIGN):
default_value = self.parse_expression ()
fields.append (ClassField (name=field_name, type_annotation=type_annotation, default=default_value, location=field_loc))
else:
self.error (f"Unexpected token in class body: {self.current ().type.name}")
self.advance ()
self.skip_newlines ()
self.expect (TokenType.RBRACE, "Expected '}' after class body")
return ClassDecl (
name=name,
parent=parent,
fields=fields,
constructor=constructor,
methods=methods,
location=loc
)
def parse_constructor (self) -> ConstructorDecl:
loc = self.location ()
self.expect (TokenType.CONSTRUCT)
self.expect (TokenType.LPAREN, "Expected '(' after 'construct'")
params = self.parse_parameters ()
self.expect (TokenType.RPAREN, "Expected ')' after parameters")
self.skip_newlines ()
body = self.parse_block ()
return ConstructorDecl (params=params, body=body, location=loc)
def parse_namespace (self) -> NamespaceDecl:
loc = self.location ()
self.expect (TokenType.NAMESPACE)
name = self.expect (TokenType.IDENTIFIER, "Expected namespace name").value
self.skip_newlines ()
self.expect (TokenType.LBRACE, "Expected '{' after namespace name")
self.skip_newlines ()
statements = []
while not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
stmt = self.parse_declaration ()
if stmt:
statements.append (stmt)
self.skip_newlines ()
self.expect (TokenType.RBRACE, "Expected '}' to close namespace")
return NamespaceDecl (name=name, statements=statements, location=loc)
def parse_using (self) -> UsingStmt:
loc = self.location ()
self.expect (TokenType.USING)
name = self.expect (TokenType.IDENTIFIER, "Expected namespace name").value
alias = None
if self.match (TokenType.ASSIGN):
alias = name
name = self.expect (TokenType.IDENTIFIER, "Expected namespace name after '='").value
names = None
if self.match (TokenType.LBRACE):
names = []
while not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
names.append (self.expect (TokenType.IDENTIFIER, "Expected symbol name").value)
if not self.match (TokenType.COMMA):
break
self.expect (TokenType.RBRACE, "Expected '}' after using names")
return UsingStmt (namespace=name, alias=alias, names=names, location=loc)
def parse_busing (self) -> BusingStmt:
loc = self.location ()
self.expect (TokenType.BUSING)
if self.check (TokenType.STRING):
path = self.advance ().value
return BusingStmt (name=None, path=path, location=loc)
name = self.expect (TokenType.IDENTIFIER, "Expected name or path").value
self.expect (TokenType.ASSIGN, "Expected '=' after busing name")
path = self.expect (TokenType.STRING, "Expected path string").value
return BusingStmt (name=name, path=path, location=loc)
def parse_statement (self) -> Optional[Statement]:
if self.check (TokenType.RETURN):
return self.parse_return ()
if self.check (TokenType.BREAK):
return self.parse_break ()
if self.check (TokenType.CONTINUE):
return self.parse_continue ()
if self.check (TokenType.IF):
return self.parse_if ()
if self.check (TokenType.WHILE):
return self.parse_while ()
if self.check (TokenType.FOR):
return self.parse_for ()
if self.check (TokenType.FOREACH):
return self.parse_foreach ()
if self.check (TokenType.WITH):
return self.parse_with ()
if self.check (TokenType.TRY):
return self.parse_try ()
if self.check (TokenType.THROW):
return self.parse_throw ()
if self.check (TokenType.DEFER):
return self.parse_defer ()
if self.check (TokenType.AWAIT):
return self.parse_await ()
if self.check (TokenType.ON):
return self.parse_on_signal ()
if self.check (TokenType.WHEN):
return self.parse_when ()
if self.check (TokenType.LBRACE):
return self.parse_block ()
return self.parse_expression_statement ()
def parse_block (self) -> Block:
loc = self.location ()
self.expect (TokenType.LBRACE, "Expected '{'")
self.skip_newlines ()
statements = []
while not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
stmt = self.parse_declaration ()
if stmt:
statements.append (stmt)
self.skip_newlines ()
self.expect (TokenType.RBRACE, "Expected '}'")
return Block (statements=statements, location=loc)
def parse_return (self) -> ReturnStmt:
loc = self.location ()
self.expect (TokenType.RETURN)
value = None
if not self.check (TokenType.NEWLINE) and not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
value = self.parse_expression ()
return ReturnStmt (value=value, location=loc)
def parse_break (self) -> BreakStmt:
loc = self.location ()
self.expect (TokenType.BREAK)
return BreakStmt (location=loc)
def parse_continue (self) -> ContinueStmt:
loc = self.location ()
self.expect (TokenType.CONTINUE)
return ContinueStmt (location=loc)
def parse_if (self) -> IfStmt:
loc = self.location ()
self.expect (TokenType.IF)
condition = self.parse_expression ()
self.skip_newlines ()
then_branch = self.parse_block ()
elif_branches = []
else_branch = None
self.skip_newlines ()
while self.check (TokenType.ELSE):
self.advance ()
if self.match (TokenType.IF):
elif_cond = self.parse_expression ()
self.skip_newlines ()
elif_block = self.parse_block ()
elif_branches.append ((elif_cond, elif_block))
self.skip_newlines ()
else:
self.skip_newlines ()
else_branch = self.parse_block ()
break
return IfStmt (
condition=condition,
then_branch=then_branch,
elif_branches=elif_branches,
else_branch=else_branch,
location=loc
)
def parse_while (self) -> WhileStmt:
loc = self.location ()
self.expect (TokenType.WHILE)
condition = self.parse_expression ()
self.skip_newlines ()
body = self.parse_block ()
return WhileStmt (condition=condition, body=body, location=loc)
def parse_for (self) -> ForStmt:
loc = self.location ()
self.expect (TokenType.FOR)
variable = self.expect (TokenType.IDENTIFIER, "Expected loop variable").value
self.expect (TokenType.IN, "Expected 'in'")
iterable = self.parse_expression ()
self.skip_newlines ()
body = self.parse_block ()
return ForStmt (variable=variable, iterable=iterable, body=body, location=loc)
def parse_foreach (self) -> ForeachStmt:
loc = self.location ()
self.expect (TokenType.FOREACH)
variables = []
variables.append (self.expect (TokenType.IDENTIFIER, "Expected loop variable").value)
if self.match (TokenType.COMMA):
variables.append (self.expect (TokenType.IDENTIFIER, "Expected second loop variable").value)
self.expect (TokenType.IN, "Expected 'in'")
iterable = self.parse_expression ()
self.skip_newlines ()
body = self.parse_block ()
return ForeachStmt (variables=variables, iterable=iterable, body=body, location=loc)
def parse_with (self) -> WithStmt:
loc = self.location ()
self.expect (TokenType.WITH)
variables = []
resources = []
variables.append (self.expect (TokenType.IDENTIFIER, "Expected variable name").value)
while self.match (TokenType.COMMA):
variables.append (self.expect (TokenType.IDENTIFIER, "Expected variable name").value)
self.expect (TokenType.IN, "Expected 'in'")
resources.append (self.parse_expression ())
while self.match (TokenType.COMMA):
resources.append (self.parse_expression ())
if len (variables) != len (resources):
self.error ("Number of variables must match number of resources in 'with' statement")
self.skip_newlines ()
body = self.parse_block ()
return WithStmt (variables=variables, resources=resources, body=body, location=loc)
def parse_try (self) -> TryStmt:
loc = self.location ()
self.expect (TokenType.TRY)
self.skip_newlines ()
try_block = self.parse_block ()
except_clauses = []
finally_block = None
self.skip_newlines ()
while self.match (TokenType.EXCEPT):
exc_type = None
exc_var = None
if self.check (TokenType.IDENTIFIER):
first_id = self.advance ().value
if self.check (TokenType.IDENTIFIER):
exc_type = first_id
exc_var = self.advance ().value
else:
exc_var = first_id
self.skip_newlines ()
exc_block = self.parse_block ()
except_clauses.append ((exc_type, exc_var, exc_block))
self.skip_newlines ()
if self.match (TokenType.FINALLY):
self.skip_newlines ()
finally_block = self.parse_block ()
return TryStmt (
try_block=try_block,
except_clauses=except_clauses,
finally_block=finally_block,
location=loc
)
def parse_throw (self) -> ThrowStmt:
loc = self.location ()
self.expect (TokenType.THROW)
expression = self.parse_expression ()
return ThrowStmt (expression=expression, location=loc)
def parse_defer (self) -> DeferStmt:
loc = self.location ()
self.expect (TokenType.DEFER)
expression = self.parse_expression ()
return DeferStmt (expression=expression, location=loc)
def parse_await (self) -> AwaitStmt:
loc = self.location ()
self.expect (TokenType.AWAIT)
expr = self.parse_expression ()
return AwaitStmt (expression=expr, location=loc)
def parse_on_signal (self) -> OnSignalStmt:
loc = self.location ()
self.expect (TokenType.ON)
signal = self.expect (TokenType.IDENTIFIER, "Expected signal name").value
self.skip_newlines ()
body = self.parse_block ()
return OnSignalStmt (signal=signal, body=body, location=loc)
def parse_when (self) -> WhenStmt:
loc = self.location ()
self.expect (TokenType.WHEN)
value = self.parse_expression ()
self.skip_newlines ()
self.expect (TokenType.LBRACE, "Expected '{' after when value")
self.skip_newlines ()
branches = []
while not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
branch = self.parse_when_branch ()
branches.append (branch)
self.skip_newlines ()
self.expect (TokenType.RBRACE, "Expected '}' after when branches")
return WhenStmt (value=value, branches=branches, location=loc)
def parse_when_branch (self) -> WhenBranch:
loc = self.location ()
if self.match (TokenType.ELSE):
self.skip_newlines ()
body = self.parse_block ()
return WhenBranch (patterns=[], is_else=True, body=body, location=loc)
patterns = []
while True:
pattern = self.parse_when_pattern ()
patterns.append (pattern)
if not self.match (TokenType.COMMA):
break
self.skip_newlines ()
body = self.parse_block ()
return WhenBranch (patterns=patterns, is_else=False, body=body, location=loc)
def parse_when_pattern (self) -> Expression:
"""Parse a single pattern: value or range (start..end)"""
loc = self.location ()
left = self.parse_primary ()
if self.match (TokenType.DOTDOT):
right = self.parse_primary ()
return RangePattern (start=left, end=right, location=loc)
return left
def parse_expression_statement (self) -> Optional[Statement]:
loc = self.location ()
expr = self.parse_expression ()
type_annotation = None
if isinstance (expr, Identifier) and self.check (TokenType.COLON):
type_annotation = self.parse_type_annotation ()
if self.check (TokenType.ASSIGN, TokenType.PLUS_ASSIGN, TokenType.MINUS_ASSIGN,
TokenType.STAR_ASSIGN, TokenType.SLASH_ASSIGN):
operator = self.advance ().value
value = self.parse_expression ()
return Assignment (target=expr, type_annotation=type_annotation, operator=operator, value=value, location=loc)
return ExpressionStmt (expression=expr, location=loc)
PRECEDENCE = {
TokenType.PIPE: 0,
TokenType.OR: 1,
TokenType.AND: 2,
TokenType.EQ: 3,
TokenType.NEQ: 3,
TokenType.LT: 4,
TokenType.GT: 4,
TokenType.LTE: 4,
TokenType.GTE: 4,
TokenType.PLUS: 5,
TokenType.MINUS: 5,
TokenType.DOTDOT: 5,
TokenType.STAR: 6,
TokenType.SLASH: 6,
TokenType.PERCENT: 6,
}
def parse_expression (self, min_prec: int = 0) -> Expression:
left = self.parse_unary ()
while True:
prec = self.PRECEDENCE.get (self.current ().type, -1)
if prec < min_prec:
break
operator = self.advance ()
right = self.parse_expression (prec + 1)
left = BinaryOp (left=left, operator=operator.value, right=right, location=left.location)
return left
def parse_unary (self) -> Expression:
if self.check (TokenType.NOT, TokenType.MINUS):
loc = self.location ()
operator = self.advance ()
operand = self.parse_unary ()
return UnaryOp (operator=operator.value, operand=operand, location=loc)
if self.check (TokenType.ASYNC):
loc = self.location ()
self.advance ()
expr = self.parse_unary ()
return AsyncExpr (expression=expr, location=loc)
if self.check (TokenType.NEW):
loc = self.location ()
self.advance () # consume 'new'
class_name = self.expect (TokenType.IDENTIFIER, "Expected class name after 'new'").value
self.expect (TokenType.LPAREN, "Expected '(' after class name")
args = []
if not self.check (TokenType.RPAREN):
args = self.parse_arguments ()
self.expect (TokenType.RPAREN, "Expected ')' after constructor arguments")
return NewExpr (class_name=class_name, arguments=args, location=loc)
return self.parse_postfix ()
def parse_postfix (self) -> Expression:
expr = self.parse_primary ()
while True:
if self.match (TokenType.LPAREN):
args = []
if not self.check (TokenType.RPAREN):
args = self.parse_arguments ()
self.expect (TokenType.RPAREN, "Expected ')' after arguments")
expr = CallExpr (callee=expr, arguments=args, location=expr.location)
elif self.match (TokenType.DOT):
member = self.expect_name ("Expected member name").value
expr = MemberAccess (object=expr, member=member, location=expr.location)
elif self.match (TokenType.LBRACKET):
index = self.parse_expression ()
self.expect (TokenType.RBRACKET, "Expected ']' after index")
expr = IndexAccess (object=expr, index=index, location=expr.location)
else:
break
return expr
def parse_arguments (self) -> List[Expression]:
args = []
while True:
args.append (self.parse_expression ())
if not self.match (TokenType.COMMA):
break
return args
def parse_primary (self) -> Expression:
loc = self.location ()
token = self.current ()
if self.match (TokenType.INTEGER):
return IntegerLiteral (value=token.value, location=loc)
if self.match (TokenType.FLOAT):
return FloatLiteral (value=token.value, location=loc)
if self.match (TokenType.STRING):
has_interp = '{' in token.value
return StringLiteral (value=token.value, has_interpolation=has_interp, location=loc)
if self.match (TokenType.TRUE):
return BoolLiteral (value=True, location=loc)
if self.match (TokenType.FALSE):
return BoolLiteral (value=False, location=loc)
if self.match (TokenType.NIL):
return NilLiteral (location=loc)
if self.match (TokenType.THIS):
return ThisExpr (location=loc)
if self.check (TokenType.BASE):
return self.parse_base_call ()
if self.match (TokenType.LBRACKET):
elements = []
if not self.check (TokenType.RBRACKET):
while True:
elements.append (self.parse_expression ())
if not self.match (TokenType.COMMA):
break
self.expect (TokenType.RBRACKET, "Expected ']'")
return ArrayLiteral (elements=elements, location=loc)
if self.check (TokenType.LBRACE):
return self.parse_dict_literal ()
if self.check (TokenType.LPAREN):
return self.parse_paren_or_lambda ()
if self.check (TokenType.IDENTIFIER):
if self.peek ().type == TokenType.ARROW:
return self.parse_lambda ()
self.advance ()
return Identifier (name=token.value, location=loc)
if self.match (TokenType.RANGE):
self.expect (TokenType.LPAREN, "Expected '(' after 'range'")
args = self.parse_arguments ()
self.expect (TokenType.RPAREN, "Expected ')'")
return CallExpr (
callee=Identifier (name="range", location=loc),
arguments=args,
location=loc
)
self.error (f"Unexpected token: {token.type.name}")
self.advance ()
return NilLiteral (location=loc)
def parse_base_call (self) -> BaseCall:
loc = self.location ()
self.expect (TokenType.BASE)
self.expect (TokenType.LPAREN, "Expected '(' after 'base'")
args = []
if not self.check (TokenType.RPAREN):
args = self.parse_arguments ()
self.expect (TokenType.RPAREN, "Expected ')'")
return BaseCall (arguments=args, location=loc)
def parse_dict_literal (self) -> DictLiteral:
loc = self.location ()
self.expect (TokenType.LBRACE)
self.skip_newlines ()
pairs = []
while not self.check (TokenType.RBRACE) and not self.check (TokenType.EOF):
key = self.parse_expression ()
self.expect (TokenType.COLON, "Expected ':' after dict key")
value = self.parse_expression ()
pairs.append ((key, value))
self.skip_newlines ()
if not self.match (TokenType.COMMA):
break
self.skip_newlines ()
self.skip_newlines ()
self.expect (TokenType.RBRACE, "Expected '}'")
return DictLiteral (pairs=pairs, location=loc)
def parse_paren_or_lambda (self) -> Expression:
loc = self.location ()
self.expect (TokenType.LPAREN)
if self.check (TokenType.RPAREN):
self.advance ()
if self.match (TokenType.ARROW):
return self.parse_lambda_body ([], loc)
self.error ("Expected '=>' after '()'")
return NilLiteral (location=loc)
if self.check (TokenType.IDENTIFIER):
saved_pos = self.pos
params = []
is_lambda = True
try:
while True:
if not self.check (TokenType.IDENTIFIER):
is_lambda = False
break
params.append (self.advance ().value)
if not self.match (TokenType.COMMA):
break
if is_lambda and self.match (TokenType.RPAREN):
if self.match (TokenType.ARROW):
return self.parse_lambda_body (params, loc)
self.pos = saved_pos
except Exception:
self.pos = saved_pos
expr = self.parse_expression ()
self.expect (TokenType.RPAREN, "Expected ')'")
return expr
def parse_lambda (self) -> Lambda:
loc = self.location ()
param = self.expect (TokenType.IDENTIFIER).value
self.expect (TokenType.ARROW)
return self.parse_lambda_body ([param], loc)
def parse_lambda_body (self, params: List[str], loc: SourceLocation) -> Lambda:
if self.check (TokenType.LBRACE):
body = self.parse_block ()
else:
body = self.parse_expression ()
return Lambda (params=params, body=body, location=loc)
bootstrap/stdlib.py deleted 100644 → 0
View file @ 5f7339cd
"""Standard library code generation.
Generates bash implementations for ContenT stdlib functions.
Uses method definitions from methods.py as single source of truth.
"""
from .methods import (
STRING_METHODS, ARRAY_METHODS, DICT_METHODS, FILE_HANDLE_METHODS,
HTTP_METHODS, FS_METHODS, JSON_METHODS, LOGGER_METHODS,
REGEX_METHODS, MATH_METHODS, TIME_METHODS, ARGS_METHODS, CORE_FUNCTIONS
)
class StdlibMixin:
"""Mixin class for standard library code generation."""
def emit_stdlib(self, used_categories: set = None):
"""Emit the ContenT standard library."""
self.emit("# === ContenT Standard Library ===")
self.emit()
self._emit_core()
if used_categories is None:
self._emit_http()
self._emit_fs()
self._emit_json()
self._emit_object_system()
self._emit_exception()
self._emit_logger()
self._emit_string()
self._emit_array()
self._emit_regex()
self._emit_utils()
self._emit_awk_wrapper()
self._emit_math()
self._emit_dict()
self._emit_misc()
else:
if 'http' in used_categories:
self._emit_http()
if 'fs' in used_categories:
self._emit_fs()
if 'json' in used_categories:
self._emit_json()
if 'object' in used_categories:
self._emit_object_system()
if 'exception' in used_categories:
self._emit_exception()
if 'logger' in used_categories:
self._emit_logger()
if 'string' in used_categories:
self._emit_string()
if 'array' in used_categories:
self._emit_array()
if 'regex' in used_categories:
self._emit_regex()
if 'args' in used_categories:
self._emit_args()
else:
self._emit_utils_minimal()
if 'awk' in used_categories or 'math' in used_categories:
self._emit_awk_wrapper()
if 'math' in used_categories:
self._emit_math()
if 'dict' in used_categories:
self._emit_dict()
if 'reflect' in used_categories:
self._emit_reflect()
if 'misc' in used_categories or 'time' in used_categories:
self._emit_misc()
if 'test' in used_categories:
self._emit_test()
self.emit("# === End Standard Library ===")
self.emit()
def _emit_core(self):
"""Core functions: print, range, len."""
self.emit("exec 3>&1 # Save stdout to FD3 for print()")
self.emit()
self.emit(f"{CORE_FUNCTIONS['print'].bash_func} () {{ {CORE_FUNCTIONS['print'].bash_impl}; }}")
self.emit()
self.emit("__ct_range () {")
with self.indented():
self.emit('local start=0 end step=1')
self.emit('case $# in')
with self.indented():
self.emit('1) end=$1 ;;')
self.emit('2) start=$1; end=$2 ;;')
self.emit('3) start=$1; end=$2; step=$3 ;;')
self.emit('esac')
self.emit('seq "$start" "$step" "$((end - 1))"')
self.emit("}")
self.emit()
self.emit(f"{CORE_FUNCTIONS['len'].bash_func} () {{ {CORE_FUNCTIONS['len'].bash_impl}; }}")
self.emit()
self.emit(f"{CORE_FUNCTIONS['pid'].bash_func} () {{ {CORE_FUNCTIONS['pid'].bash_impl}; }}")
self.emit()
def _emit_http(self):
"""HTTP functions from HTTP_METHODS."""
for method_name, method_def in HTTP_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_fs(self):
"""Filesystem functions from FS_METHODS."""
for method_name, method_def in FS_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
self._emit_file_handles()
def _emit_file_handles(self):
"""File handle functions for fs.open() / f.close()."""
self.emit("# File handle system")
self.emit("declare -gA __ct_file_handles=()")
self.emit("declare -g __ct_file_fd=10")
self.emit()
self.emit("__ct_fs_open () {")
with self.indented():
self.emit('local path="$1"')
self.emit('local mode="${2:-r}"')
self.emit('local fd=$__ct_file_fd')
self.emit('__ct_file_fd=$((fd + 1))')
self.emit('local h="__fh_${fd}"')
for key in ['path', 'fd', 'mode']:
self.emit(f'__ct_file_handles["${{h}}_{key}"]="${{{key}}}"')
self.emit('__CT_RET="$h"')
self.emit('echo "$h"')
self.emit("}")
self.emit()
for method_name, method_def in FILE_HANDLE_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
self.emit("# Context manager support")
self.emit('__ct_fh___enter__ () { echo "$1"; }')
self.emit('__ct_fh___exit__ () { __ct_fh_close "$1"; }')
self.emit()
def _emit_json(self):
"""JSON functions."""
self.emit("__ct_json_parse () {")
with self.indented():
self.emit('local __json="$1" __dict_name="$2"')
self.emit('eval "declare -gA $__dict_name=()"')
self.emit('while IFS= read -r __line; do')
with self.indented():
self.emit('local __key="${__line%%=*}" __val="${__line#*=}"')
self.emit('[[ -n "$__key" ]] && eval "${__dict_name}[\\"$__key\\"]=\"\\$__val\""')
self.emit("done < <(echo \"$__json\" | jq -r 'to_entries[] | \"\\(.key)=\\(.value)\"' 2>/dev/null)")
self.emit("}")
self.emit()
self.emit("__ct_json_get () {")
with self.indented():
self.emit('echo "$1" | jq -r "$2" 2>/dev/null')
self.emit("}")
self.emit()
self.emit("__ct_json_unmarshal () {")
with self.indented():
self.emit('local __json="$1" __class="$2"')
self.emit('"$__class"')
self.emit('local __obj="$__ct_last_instance"')
self.emit('local __cls="${__ct_obj_class[$__obj]}"')
self.emit('local -n __fields="__ct_class_meta_${__cls}_fields"')
self.emit('local -n __types="__ct_class_meta_${__cls}_types"')
self.emit('local __f')
self.emit('for __f in "${__fields[@]}"; do')
with self.indented():
self.emit('local __t="${__types[$__f]}"')
self.emit('local __val')
self.emit('if declare -p "__ct_class_meta_${__t}_fields" &>/dev/null; then')
with self.indented():
self.emit('__val="$(echo "$__json" | jq -c --arg f "$__f" \'.[$f] // empty\' 2>/dev/null)"')
self.emit('if [[ -n "$__val" && "$__val" != "null" ]]; then')
with self.indented():
self.emit('__ct_json_unmarshal "$__val" "$__t"')
self.emit('__CT_OBJ["$__obj.$__f"]="$__ct_last_instance"')
self.emit('fi')
self.emit('else')
with self.indented():
self.emit('__val="$(echo "$__json" | jq -r --arg f "$__f" \'.[$f] // empty\' 2>/dev/null)"')
self.emit('if [[ -n "$__val" ]]; then')
with self.indented():
self.emit('__CT_OBJ["$__obj.$__f"]="$__val"')
self.emit('fi')
self.emit('fi')
self.emit('done')
self.emit('__ct_last_instance="$__obj"')
self.emit("}")
self.emit()
self.emit("__ct_json_marshal () {")
with self.indented():
self.emit('local __obj="$1"')
self.emit('local __cls="${__ct_obj_class[$__obj]}"')
self.emit('local -n __fields="__ct_class_meta_${__cls}_fields"')
self.emit('local -n __types="__ct_class_meta_${__cls}_types"')
self.emit('local __first=1')
self.emit('printf "{"')
self.emit('for __f in "${__fields[@]}"; do')
with self.indented():
self.emit('[[ $__first -eq 1 ]] && __first=0 || printf ","')
self.emit('printf "\\"%s\\":" "$__f"')
self.emit('local __v="${__CT_OBJ["$__obj.$__f"]}"')
self.emit('local __t="${__types[$__f]}"')
self.emit('if [[ -n "${__ct_obj_class[$__v]+x}" ]]; then')
with self.indented():
self.emit('__ct_json_marshal "$__v"')
self.emit('elif [[ "$__t" == "int" || "$__t" == "float" || "$__t" == "bool" ]]; then')
with self.indented():
self.emit('printf "%s" "$__v"')
self.emit('else')
with self.indented():
self.emit('printf "\\"%s\\"" "$__v"')
self.emit('fi')
self.emit('done')
self.emit('printf "}"')
self.emit("}")
self.emit()
self.emit("__ct_json_stringify () {")
with self.indented():
self.emit('local -n __d="$1"')
self.emit('local __first=1')
self.emit('printf "{"')
self.emit('for __k in "${!__d[@]}"; do')
with self.indented():
self.emit('[[ $__first -eq 1 ]] && __first=0 || printf ","')
self.emit('printf "\\"%s\\":" "$__k"')
self.emit('local __v="${__d[$__k]}"')
self.emit('if [[ "$__v" =~ ^-?[0-9]+$ ]] || [[ "$__v" =~ ^(true|false|null)$ ]]; then')
with self.indented():
self.emit('printf "%s" "$__v"')
self.emit('else')
with self.indented():
self.emit('printf "\\"%s\\"" "$__v"')
self.emit('fi')
self.emit('done')
self.emit('printf "}\\n"')
self.emit("}")
self.emit()
def _emit_object_system(self):
"""Object system: class tracking, method dispatch."""
self.emit("declare -gA __ct_obj_class=()")
self.emit("declare -gA __CT_OBJ=()")
self.emit("declare -g __ct_last_instance=''")
self.emit()
self.emit("__ct_call_method () {")
with self.indented():
self.emit('local __obj="$1" __method="$2"')
self.emit('shift 2')
self.emit('local __class="${__ct_obj_class[$__obj]:-}"')
self.emit('[[ -z "$__class" ]] && { echo "Error: Unknown object $__obj" >&2; return 1; }')
self.emit('"__ct_class_${__class}_${__method}" "$__obj" "$@"')
self.emit("}")
self.emit()
self.emit('__ct_get_field () { echo "${__CT_OBJ[$1.$2]}"; }')
self.emit()
self.emit("declare -g __CT_RET=''")
self.emit()
def _emit_exception(self):
"""Exception handling."""
self.emit("declare -g __ct_exception=''")
self.emit("declare -g __ct_exception_type=''")
self.emit()
self.emit("__ct_throw () {")
with self.indented():
self.emit('__ct_exception_type="$1"')
self.emit('__ct_exception="$2"')
self.emit('return 1')
self.emit("}")
self.emit()
def _emit_logger(self):
"""Logger functions from LOGGER_METHODS."""
for method_name, method_def in LOGGER_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_string(self):
"""String functions from STRING_METHODS."""
self.emit("# String functions")
for method_name, method_def in STRING_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_array(self):
"""Array functions from ARRAY_METHODS."""
self.emit("# Array functions")
for method_name, method_def in ARRAY_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
self.emit("# Array map/filter with lambda functions")
self.emit("__ct_arr_map () {")
with self.indented():
self.emit('local -n __a=$1')
self.emit('local __fn=$2')
self.emit('local -a __result=()')
self.emit('for __item in "${__a[@]}"; do __result+=("$($__fn "$__item")"); done')
self.emit('__CT_RET=("${__result[@]}")')
self.emit("}")
self.emit()
self.emit("__ct_arr_filter () {")
with self.indented():
self.emit('local -n __a=$1')
self.emit('local __fn=$2')
self.emit('local -a __result=()')
self.emit('for __item in "${__a[@]}"; do')
with self.indented():
self.emit('[[ "$($__fn "$__item")" == "true" ]] && __result+=("$__item")')
self.emit('done')
self.emit('__CT_RET=("${__result[@]}")')
self.emit("}")
self.emit()
self.emit("# Fast array functions (no echo)")
self.emit("__ct_arr_len_fast () { local -n __a=$1; __CT_RET=${#__a[@]}; }")
self.emit("__ct_arr_get_fast () { local -n __a=$1; __CT_RET=\"${__a[$2]}\"; }")
self.emit()
def _emit_regex(self):
"""Regex functions from REGEX_METHODS."""
self.emit("# Regex functions")
for method_name, method_def in REGEX_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_utils(self):
"""Utility functions."""
self.emit(f"{CORE_FUNCTIONS['exit'].bash_func} () {{ {CORE_FUNCTIONS['exit'].bash_impl}; }}")
self.emit()
self.emit(f"{CORE_FUNCTIONS['is_number'].bash_func} () {{ {CORE_FUNCTIONS['is_number'].bash_impl}; }}")
self.emit(f"{CORE_FUNCTIONS['is_empty'].bash_func} () {{ {CORE_FUNCTIONS['is_empty'].bash_impl}; }}")
self.emit()
self.emit('__ct_args=("$@")')
for method_name, method_def in ARGS_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_utils_minimal(self):
"""Minimal utility functions (without args)."""
self.emit(f"{CORE_FUNCTIONS['exit'].bash_func} () {{ {CORE_FUNCTIONS['exit'].bash_impl}; }}")
self.emit()
self.emit(f"{CORE_FUNCTIONS['is_number'].bash_func} () {{ {CORE_FUNCTIONS['is_number'].bash_impl}; }}")
self.emit(f"{CORE_FUNCTIONS['is_empty'].bash_func} () {{ {CORE_FUNCTIONS['is_empty'].bash_impl}; }}")
self.emit()
def _emit_args(self):
"""Args functions only."""
self._emit_utils()
def _emit_awk_wrapper(self):
"""AWK wrapper functions."""
self.emit("# AWK wrapper - uses mawk if available (2-10x faster)")
self.emit("__ct_awk_cmd=''")
self.emit("if command -v mawk &>/dev/null; then __ct_awk_cmd=mawk")
self.emit("elif command -v gawk &>/dev/null; then __ct_awk_cmd=gawk; echo '[WARN] mawk not found, using gawk (slower)' >&2")
self.emit("elif command -v awk &>/dev/null; then __ct_awk_cmd=awk; echo '[WARN] mawk not found, using awk (slower)' >&2")
self.emit("else echo '[ERROR] No awk implementation found' >&2; exit 1; fi")
self.emit('__ct_awk () { "$__ct_awk_cmd" "$@"; }')
self.emit('__ct_awk_file () { "$__ct_awk_cmd" -f "$1" "$2"; }')
self.emit()
def _emit_math(self):
"""Math functions from MATH_METHODS."""
self.emit("# Math functions")
for method_name, method_def in MATH_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_reflect(self):
self.emit("# Reflect functions")
self.emit("__ct_reflect_fields () {")
with self.indented():
self.emit('local __obj="$1"')
self.emit('local __cls="${__ct_obj_class[$__obj]}"')
self.emit('local -n __fields="__ct_class_meta_${__cls}_fields"')
self.emit('__CT_RET_ARR=("${__fields[@]}")')
self.emit("}")
self.emit()
self.emit("__ct_reflect_type () {")
with self.indented():
self.emit('local __obj="$1" __field="$2"')
self.emit('local __cls="${__ct_obj_class[$__obj]}"')
self.emit('local -n __types="__ct_class_meta_${__cls}_types"')
self.emit('__CT_RET="${__types[$__field]}"')
self.emit('echo "${__types[$__field]}"')
self.emit("}")
self.emit()
self.emit("__ct_reflect_get () {")
with self.indented():
self.emit('local __obj="$1" __field="$2"')
self.emit('__CT_RET="${__CT_OBJ["$__obj.$__field"]}"')
self.emit('echo "${__CT_OBJ["$__obj.$__field"]}"')
self.emit("}")
self.emit()
self.emit("__ct_reflect_set () {")
with self.indented():
self.emit('local __obj="$1" __field="$2" __value="$3"')
self.emit('__CT_OBJ["$__obj.$__field"]="$__value"')
self.emit("}")
self.emit()
self.emit("__ct_reflect_class_name () {")
with self.indented():
self.emit('local __obj="$1"')
self.emit('__CT_RET="${__ct_obj_class[$__obj]}"')
self.emit('echo "${__ct_obj_class[$__obj]}"')
self.emit("}")
self.emit()
self.emit("__ct_reflect_create () {")
with self.indented():
self.emit('local __class="$1"; shift')
self.emit('"$__class" "$@"')
self.emit("}")
self.emit()
def _emit_dict(self):
"""Dict functions from DICT_METHODS."""
self.emit("# Dict functions")
for method_name, method_def in DICT_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
def _emit_misc(self):
"""Miscellaneous functions."""
self.emit("# Byte/binary functions")
self.emit("__ct_byte_to_hex () { printf '%02x' \"$1\"; }")
self.emit("__ct_hex_to_byte () { printf '%d' \"0x$1\"; }")
self.emit()
self.emit("# Time functions")
for method_name, method_def in TIME_METHODS.items():
if method_def.bash_impl:
self.emit(f"{method_def.bash_func} () {{ {method_def.bash_impl}; }}")
self.emit()
self.emit("# Random")
self.emit(f"{CORE_FUNCTIONS['random'].bash_func} () {{ {CORE_FUNCTIONS['random'].bash_impl}; }}")
self.emit(f"{CORE_FUNCTIONS['random_range'].bash_func} () {{ {CORE_FUNCTIONS['random_range'].bash_impl}; }}")
self.emit()
self.emit("__CT_NL=$'\\n'")
self.emit()
def _emit_test(self):
"""Test framework functions."""
self.emit("# Test framework")
self.emit("declare -g __ct_test_passed=0")
self.emit("declare -g __ct_test_failed=0")
self.emit("declare -g __ct_test_current=''")
self.emit("declare -g __ct_test_start_time=0")
self.emit()
self.emit("__ct_test_colors () {")
with self.indented():
self.emit('if [[ -t 1 ]]; then')
with self.indented():
for c in ['GREEN=32', 'RED=31', 'YELLOW=33', 'CYAN=36']:
name, code = c.split('=')
self.emit(f'__CT_{name}="\\033[{code}m"')
self.emit('__CT_RESET="\\033[0m"')
self.emit('else')
with self.indented():
self.emit('__CT_GREEN="" __CT_RED="" __CT_YELLOW="" __CT_CYAN="" __CT_RESET=""')
self.emit('fi')
self.emit("}")
self.emit("__ct_test_colors")
self.emit()
self.emit("__ct_test_start () {")
with self.indented():
self.emit('__ct_test_current="$1"')
self.emit('__ct_test_start_time=$(date +%s%3N)')
self.emit('printf "${__CT_CYAN}RUN${__CT_RESET} %s\\n" "$1"')
self.emit("}")
self.emit()
self.emit("__ct_test_pass () {")
with self.indented():
self.emit('local elapsed=$(($(date +%s%3N) - __ct_test_start_time))')
self.emit('printf "${__CT_GREEN}PASS${__CT_RESET} %s ${__CT_YELLOW}(%dms)${__CT_RESET}\\n" "$__ct_test_current" "$elapsed"')
self.emit('((__ct_test_passed++)) || true')
self.emit("}")
self.emit()
self.emit("__ct_test_fail () {")
with self.indented():
self.emit('local msg="$1"')
self.emit('local elapsed=$(($(date +%s%3N) - __ct_test_start_time))')
self.emit('printf "${__CT_RED}FAIL${__CT_RESET} %s ${__CT_YELLOW}(%dms)${__CT_RESET}\\n" "$__ct_test_current" "$elapsed"')
self.emit('[[ -n "$msg" ]] && printf " ${__CT_RED}%s${__CT_RESET}\\n" "$msg"')
self.emit('((__ct_test_failed++)) || true')
self.emit("}")
self.emit()
self.emit("__ct_assert () {")
with self.indented():
self.emit('local condition="$1" msg="${2:-Assertion failed}"')
self.emit('if eval "$condition"; then return 0')
self.emit('else __ct_test_fail "$msg"; return 1; fi')
self.emit("}")
self.emit()
self.emit("__ct_assert_eq () {")
with self.indented():
self.emit('local actual="$1" expected="$2"')
self.emit('[[ "$actual" != "$expected" ]] && { __ct_test_fail "Expected \'$expected\' but got \'$actual\'"; return 1; }')
self.emit("}")
self.emit()
self.emit("__ct_test_summary () {")
with self.indented():
self.emit('echo')
self.emit('local total=$((__ct_test_passed + __ct_test_failed))')
self.emit('if [[ $__ct_test_failed -eq 0 ]]; then')
with self.indented():
self.emit('printf "${__CT_GREEN}All %d tests passed${__CT_RESET}\\n" "$total"')
self.emit('else')
with self.indented():
self.emit('printf "${__CT_RED}%d of %d tests failed${__CT_RESET}\\n" "$__ct_test_failed" "$total"')
self.emit('fi')
self.emit('return $__ct_test_failed')
self.emit("}")
self.emit()
bootstrap/stmt_codegen.py deleted 100644 → 0
View file @ 5f7339cd
from .ast_nodes import (
FunctionDecl, ClassDecl, Assignment, ExpressionStmt, IfStmt,
WhileStmt, ForStmt, ForeachStmt, WithStmt, TryStmt, ThrowStmt, DeferStmt,
AwaitStmt, OnSignalStmt, AsyncExpr,
WhenStmt, RangePattern, ReturnStmt, BreakStmt, ContinueStmt, Block,
CallExpr, Identifier, MemberAccess, ThisExpr, StringLiteral, NewExpr,
BinaryOp, DictLiteral, ArrayLiteral, WhenBranch,
NamespaceDecl, UsingStmt, BusingStmt
)
from .constants import RET_VAR, RET_ARR, COPROC_PREFIX
class StmtMixin:
"""Mixin for statement generation."""
def generate_statement(self, stmt):
if isinstance(stmt, FunctionDecl):
self.generate_function(stmt)
elif isinstance(stmt, ClassDecl):
if self.current_namespace:
self.generate_class(stmt)
elif self.used_classes is not None and stmt.name not in self.used_classes:
self.emit(f"# DCE: skipped unused class {stmt.name}")
else:
self.generate_class(stmt)
elif isinstance(stmt, NamespaceDecl):
self.generate_namespace(stmt)
elif isinstance(stmt, UsingStmt):
self.generate_using(stmt)
elif isinstance(stmt, BusingStmt):
self.generate_busing(stmt)
elif isinstance(stmt, Assignment):
self.generate_assignment(stmt)
elif isinstance(stmt, ExpressionStmt):
self.generate_expression_stmt(stmt)
elif isinstance(stmt, IfStmt):
self.generate_if(stmt)
elif isinstance(stmt, WhileStmt):
self.generate_while(stmt)
elif isinstance(stmt, ForStmt):
self.generate_for(stmt)
elif isinstance(stmt, ForeachStmt):
self.generate_foreach(stmt)
elif isinstance(stmt, WithStmt):
self.generate_with(stmt)
elif isinstance(stmt, TryStmt):
self.generate_try(stmt)
elif isinstance(stmt, ThrowStmt):
self.generate_throw(stmt)
elif isinstance(stmt, DeferStmt):
self.generate_defer(stmt)
elif isinstance(stmt, AwaitStmt):
self.generate_await(stmt)
elif isinstance(stmt, OnSignalStmt):
self.generate_on_signal(stmt)
elif isinstance(stmt, WhenStmt):
self.generate_when(stmt)
elif isinstance(stmt, ReturnStmt):
self.generate_return(stmt)
elif isinstance(stmt, BreakStmt):
self.emit("break")
elif isinstance(stmt, ContinueStmt):
self.emit("continue")
elif isinstance(stmt, Block):
for s in stmt.statements:
self.generate_statement(s)
def generate_namespace(self, stmt: NamespaceDecl):
old_ns = self.current_namespace
self.current_namespace = stmt.name
for s in stmt.statements:
self.generate_statement(s)
self.current_namespace = old_ns
def generate_using(self, stmt: UsingStmt):
pass
def generate_busing(self, stmt: BusingStmt):
pass
def generate_if(self, stmt: IfStmt):
if isinstance(stmt.condition, CallExpr) and isinstance(stmt.condition.callee, Identifier):
func_name = stmt.condition.callee.name
resolved = self._resolve_name(func_name)
if resolved in self.functions or func_name in getattr(self, 'callback_vars', set()):
args = [self.generate_expr(a) for a in stmt.condition.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'{resolved} {args_str} >/dev/null')
self.emit(f'if [[ "${{{RET_VAR}}}" == "true" ]]; then')
else:
cond = self.generate_condition(stmt.condition)
self.emit(f"if {cond}; then")
else:
mapping, _ = self.precompute_all_calls(stmt.condition)
if mapping:
cond = self.generate_condition_with_precompute(stmt.condition, mapping)
else:
cond = self.generate_condition(stmt.condition)
self.emit(f"if {cond}; then")
with self.indented():
for s in stmt.then_branch.statements:
self.generate_statement(s)
for elif_cond, elif_block in stmt.elif_branches:
cond = self.generate_condition(elif_cond)
self.emit(f"elif {cond}; then")
with self.indented():
for s in elif_block.statements:
self.generate_statement(s)
if stmt.else_branch:
self.emit("else")
with self.indented():
for s in stmt.else_branch.statements:
self.generate_statement(s)
self.emit("fi")
def generate_when(self, stmt: WhenStmt):
value_expr = self.generate_expr(stmt.value)
has_ranges = any(
any(isinstance(p, RangePattern) for p in branch.patterns)
for branch in stmt.branches if not branch.is_else
)
if has_ranges:
self._generate_when_if_chain(stmt, value_expr)
else:
self._generate_when_case(stmt, value_expr)
def _generate_when_case(self, stmt: WhenStmt, value_expr: str):
self.emit(f'case "{value_expr}" in')
with self.indented():
for branch in stmt.branches:
if branch.is_else:
self.emit("*)")
else:
patterns = []
for p in branch.patterns:
val = self.generate_expr(p)
if isinstance(p, StringLiteral):
patterns.append(f'"{val}"' if val else '""')
else:
patterns.append(val)
self.emit(f'{"|".join(patterns)})')
with self.indented():
for s in branch.body.statements:
self.generate_statement(s)
self.emit(";;")
self.emit("esac")
def _generate_when_if_chain(self, stmt: WhenStmt, value_expr: str):
self.emit(f'__when_val="{value_expr}"')
first = True
for branch in stmt.branches:
if branch.is_else:
self.emit("else")
else:
conditions = []
for p in branch.patterns:
if isinstance(p, RangePattern):
start = self.generate_expr(p.start)
end = self.generate_expr(p.end)
conditions.append(f'(( __when_val >= {start} && __when_val <= {end} ))')
else:
val = self.generate_expr(p)
conditions.append(f'[[ "$__when_val" == "{val}" ]]')
cond_str = " || ".join(conditions)
if first:
self.emit(f"if {cond_str}; then")
first = False
else:
self.emit(f"elif {cond_str}; then")
with self.indented():
for s in branch.body.statements:
self.generate_statement(s)
self.emit("fi")
def generate_while(self, stmt: WhileStmt):
mapping, regen_code = self.precompute_all_calls(stmt.condition)
if mapping:
cond = self.generate_condition_with_precompute(stmt.condition, mapping)
self.emit(f"while {cond}; do")
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
if regen_code:
self.emit("# CSE: re-compute condition vars")
for call_line, assign_line in regen_code:
self.emit(call_line)
if assign_line:
self.emit(assign_line)
self.emit("done")
return
if isinstance(stmt.condition, CallExpr) and isinstance(stmt.condition.callee, Identifier):
func_name = stmt.condition.callee.name
resolved = self._resolve_name(func_name)
if resolved in self.functions or func_name in getattr(self, 'callback_vars', set()):
args = [self.generate_expr(a) for a in stmt.condition.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit("while true; do")
with self.indented():
self.emit(f'{resolved} {args_str} >/dev/null')
self.emit(f'if [[ "${{{RET_VAR}}}" != "true" ]]; then break; fi')
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
return
cond = self.generate_condition(stmt.condition)
self.emit(f"while {cond}; do")
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
def generate_for(self, stmt: ForStmt):
var = stmt.variable
if isinstance(stmt.iterable, CallExpr) and isinstance(stmt.iterable.callee, Identifier):
if stmt.iterable.callee.name == "range":
args = [self.generate_expr(a) for a in stmt.iterable.arguments]
if len(args) == 1:
self.emit(f"for {var} in $(seq 0 $(({args[0]} - 1))); do")
elif len(args) == 2:
self.emit(f"for {var} in $(seq {args[0]} $(({args[1]} - 1))); do")
else:
self.emit(f"for {var} in $(seq {args[0]} {args[2]} $(({args[1]} - 1))); do")
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
return
iterable = self.generate_expr(stmt.iterable)
self.emit(f'for {var} in {iterable}; do')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
def generate_foreach(self, stmt: ForeachStmt):
if isinstance(stmt.iterable, CallExpr):
if isinstance(stmt.iterable.callee, Identifier) and stmt.iterable.callee.name == "range":
args = [self.generate_expr(a) for a in stmt.iterable.arguments]
var = stmt.variables[0]
if len(args) == 1:
self.emit(f"for {var} in $(seq 0 $(({args[0]} - 1))); do")
elif len(args) == 2:
self.emit(f"for {var} in $(seq {args[0]} $(({args[1]} - 1))); do")
else:
self.emit(f"for {var} in $(seq {args[0]} {args[2]} $(({args[1]} - 1))); do")
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
return
if isinstance(stmt.iterable, Identifier):
arr_name = stmt.iterable.name
if arr_name in getattr(self, 'process_handle_vars', set()):
cp = self.process_handle_map[arr_name]
var = stmt.variables[0]
self.emit(f'while read -r {var} <&${cp}_rd; do')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
return
param_map = getattr(self, 'param_name_map', {})
arr_name = param_map.get(arr_name, arr_name)
if len(stmt.variables) == 1:
var = stmt.variables[0]
self.emit(f'for {var} in "${{{arr_name}[@]}}"; do')
else:
idx_var = stmt.variables[0]
val_var = stmt.variables[1]
self.emit(f'{idx_var}=0')
self.emit(f'for {val_var} in "${{{arr_name}[@]}}"; do')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
if len(stmt.variables) == 2:
self.emit(f'((++{stmt.variables[0]}))')
self.emit("done")
return
if isinstance(stmt.iterable, CallExpr):
if isinstance(stmt.iterable.callee, MemberAccess):
if (isinstance(stmt.iterable.callee.object, Identifier) and
stmt.iterable.callee.object.name == "str" and
stmt.iterable.callee.member == "split" and
len(stmt.iterable.arguments) == 2):
str_arg = self.generate_expr(stmt.iterable.arguments[0])
delim_arg = self.generate_expr(stmt.iterable.arguments[1])
var = stmt.variables[0]
self.emit(f'IFS=\'{delim_arg}\' read -ra __ct_split_arr <<< "{str_arg}"')
if len(stmt.variables) == 1:
self.emit(f'for {var} in "${{__ct_split_arr[@]}}"; do')
else:
idx_var = stmt.variables[0]
val_var = stmt.variables[1]
self.emit(f'{idx_var}=0')
self.emit(f'for {val_var} in "${{__ct_split_arr[@]}}"; do')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
if len(stmt.variables) == 2:
self.emit(f'((++{stmt.variables[0]}))')
self.emit("done")
return
if stmt.iterable.callee.member == "split" and len(stmt.iterable.arguments) == 1:
str_expr = self.generate_expr(stmt.iterable.callee.object)
delim_arg = self.generate_expr(stmt.iterable.arguments[0])
var = stmt.variables[0]
self.emit(f'__ct_str_split "{str_expr}" "{delim_arg}"')
if len(stmt.variables) == 1:
self.emit(f'for {var} in "${{{RET_ARR}[@]}}"; do')
else:
idx_var = stmt.variables[0]
val_var = stmt.variables[1]
self.emit(f'{idx_var}=0')
self.emit(f'for {val_var} in "${{{RET_ARR}[@]}}"; do')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
if len(stmt.variables) == 2:
self.emit(f'((++{stmt.variables[0]}))')
self.emit("done")
return
iterable = self.generate_expr(stmt.iterable)
var = stmt.variables[0]
self.emit(f'for {var} in {iterable}; do')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("done")
def generate_with(self, stmt: WithStmt):
self.emit("# with statement")
local_kw = "local " if self.in_function else ""
with_async = {}
for i, (var, resource) in enumerate(zip(stmt.variables, stmt.resources)):
if isinstance(resource, AsyncExpr):
self.coproc_counter += 1
cp_name = f'{COPROC_PREFIX}{self.coproc_counter}'
inner = resource.expression
if isinstance(inner, CallExpr) and self._is_shell_command(inner):
cmd_str = self._extract_shell_command_str(inner)
elif isinstance(inner, CallExpr) and isinstance(inner.callee, Identifier):
func_name = inner.callee.name
args_str = self._generate_call_args_str(inner.arguments)
cmd_str = f'exec 3>&1; {func_name} {args_str}'.strip()
else:
cmd_str = self.generate_expr(inner)
self.emit(f'coproc {cp_name} {{ {cmd_str}; }}')
self.emit(f'exec {{{cp_name}_wr}}>&${{{cp_name}[1]}}')
self.emit(f'exec {{{cp_name}_rd}}<&${{{cp_name}[0]}}')
self.emit(f'{cp_name}_pid=${cp_name}_PID')
self.emit(f'eval "exec ${{{cp_name}[1]}}>&-"')
self.process_handle_vars.add(var)
self.process_handle_map[var] = cp_name
with_async[i] = cp_name
continue
if isinstance(resource, CallExpr) and isinstance(resource.callee, MemberAccess):
if resource.callee.member == "open":
self.file_handle_vars.add(var)
args = [self.generate_expr(a) for a in resource.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'__ct_fs_open {args_str} >/dev/null')
self.emit(f'{local_kw}__ct_with_{i}="${{{RET_VAR}}}"')
self.emit(f'{local_kw}{var}="$__ct_with_{i}"')
continue
res_expr = self.generate_call_statement(resource) if isinstance(resource, CallExpr) else self.generate_expr(resource)
self.emit(f'{local_kw}__ct_with_{i}=$({res_expr})')
self.emit(f'{local_kw}{var}="$__ct_with_{i}"')
for s in stmt.body.statements:
self.generate_statement(s)
self.emit("# with cleanup")
for i in range(len(stmt.variables) - 1, -1, -1):
if i in with_async:
cp = with_async[i]
self.emit(f'exec {{{cp}_wr}}>&- 2>/dev/null || true')
self.emit(f'wait ${cp}_pid 2>/dev/null || true')
else:
self.emit(f'__ct_fh___exit__ "$__ct_with_{i}"')
def generate_try(self, stmt: TryStmt):
self.emit("# try/except block")
self.emit("set +e")
self.emit("__ct_exception=''")
self.emit("__ct_exception_type=''")
self.emit("__ct_try_failed=0")
self.emit()
self.emit("while true; do")
with self.indented():
for s in stmt.try_block.statements:
self.generate_statement(s)
self.emit('if [[ $? -ne 0 ]]; then')
with self.indented():
self.emit('__ct_try_failed=1')
self.emit('__ct_exception_type="Error"')
self.emit('__ct_exception="Command failed"')
self.emit('break')
self.emit('fi')
self.emit("break")
self.emit("done")
self.emit()
self.emit("set -e")
self.emit()
self.emit('if [[ "$__ct_try_failed" == "1" ]]; then')
with self.indented():
for i, (exc_type, exc_var, exc_block) in enumerate(stmt.except_clauses):
if i == 0:
if exc_type:
self.emit(f'if [[ "$__ct_exception_type" == "{exc_type}" ]]; then')
else:
if exc_type:
self.emit(f'elif [[ "$__ct_exception_type" == "{exc_type}" ]]; then')
else:
self.emit("else")
if exc_type or i > 0:
with self.indented():
if exc_var:
self.emit(f'{exc_var}="$__ct_exception"')
for s in exc_block.statements:
self.generate_statement(s)
else:
if exc_var:
self.emit(f'{exc_var}="$__ct_exception"')
for s in exc_block.statements:
self.generate_statement(s)
if stmt.except_clauses and stmt.except_clauses[0][0] is not None:
self.emit("fi")
self.emit("fi")
if stmt.finally_block:
self.emit()
self.emit("# finally block")
for s in stmt.finally_block.statements:
self.generate_statement(s)
def generate_throw(self, stmt: ThrowStmt):
if isinstance(stmt.expression, CallExpr):
if isinstance(stmt.expression.callee, Identifier):
exc_type = stmt.expression.callee.name
exc_args = [self.generate_expr(a) for a in stmt.expression.arguments]
exc_msg = exc_args[0] if exc_args else ""
self.emit(f'__ct_throw "{exc_type}" "{exc_msg}"')
return
expr = self.generate_expr(stmt.expression)
self.emit(f'__ct_throw "Error" "{expr}"')
def generate_defer(self, stmt: DeferStmt):
if isinstance(stmt.expression, CallExpr):
call = self.generate_call_statement(stmt.expression)
self.deferred_calls.append(call)
else:
expr = self.generate_expr(stmt.expression)
self.deferred_calls.append(expr)
def generate_await(self, stmt: AwaitStmt):
if isinstance(stmt.expression, Identifier):
var = stmt.expression.name
if var in self.process_handle_vars:
cp = self.process_handle_map[var]
self.emit(f'wait ${cp}_pid 2>/dev/null || true')
return
expr = self.generate_expr(stmt.expression)
self.emit(f'wait "{expr}" 2>/dev/null || true')
def generate_on_signal(self, stmt: OnSignalStmt):
signal_map = {
"SIGINT": "INT", "SIGTERM": "TERM", "SIGHUP": "HUP",
"SIGUSR1": "USR1", "SIGUSR2": "USR2", "EXIT": "EXIT",
}
sig = signal_map.get(stmt.signal, stmt.signal)
handler = f'__ct_on_{sig.lower()}'
self.emit(f'{handler} () {{')
with self.indented():
for s in stmt.body.statements:
self.generate_statement(s)
self.emit('}')
self.emit(f'trap {handler} {sig}')
def generate_return(self, stmt: ReturnStmt):
if self.deferred_calls:
self.emit("# Deferred calls before return")
for call in reversed(self.deferred_calls):
self.emit(call)
if stmt.value:
if isinstance(stmt.value, CallExpr) and isinstance(stmt.value.callee, MemberAccess):
if isinstance(stmt.value.callee.object, Identifier) and stmt.value.callee.object.name == "str":
result = self._generate_inline_str_return(stmt.value)
if result:
return
if isinstance(stmt.value, CallExpr):
result = self._generate_call_return(stmt.value)
if result:
return
if isinstance(stmt.value, NewExpr):
resolved_class = self._resolve_name(stmt.value.class_name)
args = [self.generate_expr(arg) for arg in stmt.value.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'{resolved_class} {args_str}')
self.emit(f'{RET_VAR}="$__ct_last_instance"')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return
if isinstance(stmt.value, BinaryOp) and stmt.value.operator in ("==", "!=", "<", ">", "<=", ">=", "&&", "||"):
cond = self.generate_condition(stmt.value)
self.emit(f'{cond} && {RET_VAR}=true || {RET_VAR}=false')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return
if isinstance(stmt.value, DictLiteral):
self.emit('local __ct_ret_dict="__ct_dict_$RANDOM$RANDOM"')
self.emit('declare -gA "$__ct_ret_dict"')
for k, v in stmt.value.pairs:
key = self.generate_expr(k)
val = self.generate_expr(v)
self.emit(f'eval "$__ct_ret_dict[{key}]=\\"{val}\\""')
self.emit('{RET_VAR}="$__ct_ret_dict"')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return
if isinstance(stmt.value, ArrayLiteral):
self.emit('local __ct_ret_arr="__ct_arr_$RANDOM$RANDOM"')
self.emit('declare -ga "$__ct_ret_arr"')
elements = [self.generate_expr(e) for e in stmt.value.elements]
for i, elem in enumerate(elements):
self.emit(f'eval "$__ct_ret_arr[{i}]=\\"{elem}\\""')
self.emit('{RET_VAR}="$__ct_ret_arr"')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return
value = self.generate_expr(stmt.value)
self.emit(f'{RET_VAR}="{value}"')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
else:
self.emit("return 0")
def _generate_inline_str_return(self, expr: CallExpr) -> bool:
"""Inline str.contains/starts/ends for returns."""
method = expr.callee.member
if method == "contains" and len(expr.arguments) == 2:
haystack = self.generate_expr(expr.arguments[0])
needle = self.generate_expr(expr.arguments[1])
self.emit(f'[[ "{haystack}" == *"{needle}"* ]] && {RET_VAR}=true || {RET_VAR}=false')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif method == "starts" and len(expr.arguments) == 2:
s = self.generate_expr(expr.arguments[0])
prefix = self.generate_expr(expr.arguments[1])
self.emit(f'[[ "{s}" == "{prefix}"* ]] && {RET_VAR}=true || {RET_VAR}=false')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif method == "ends" and len(expr.arguments) == 2:
s = self.generate_expr(expr.arguments[0])
suffix = self.generate_expr(expr.arguments[1])
self.emit(f'[[ "{s}" == *"{suffix}" ]] && {RET_VAR}=true || {RET_VAR}=false')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
return False
def _generate_call_return(self, expr: CallExpr) -> bool:
"""Generate return for function/method calls."""
if isinstance(expr.callee, MemberAccess):
if isinstance(expr.callee.object, MemberAccess) and isinstance(expr.callee.object.object, ThisExpr):
field = expr.callee.object.member
method = expr.callee.member
args = [self.generate_expr(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
field_type = self.class_field_types.get((self.current_class, field)) if self.current_class else None
arr_methods = {
"len": "__ct_arr_len", "push": "__ct_arr_push", "pop": "__ct_arr_pop",
"shift": "__ct_arr_shift", "join": "__ct_arr_join", "get": "__ct_arr_get",
"set": "__ct_arr_set", "slice": "__ct_arr_slice",
}
dict_methods = {
"get": "__ct_dict_get", "set": "__ct_dict_set", "has": "__ct_dict_has",
"del": "__ct_dict_del", "keys": "__ct_dict_keys",
}
str_methods = {
"len": "__ct_str_len", "upper": "__ct_str_upper", "lower": "__ct_str_lower",
"trim": "__ct_str_trim", "contains": "__ct_str_contains", "starts": "__ct_str_starts",
"ends": "__ct_str_ends", "index": "__ct_str_index", "replace": "__ct_str_replace",
"substr": "__ct_str_substr", "split": "__ct_str_split", "charAt": "__ct_str_char_at",
}
if field_type == "array" and method in arr_methods:
func_name = arr_methods[method]
self.emit(f'{func_name} "${{this}}_{field}" {args_str} >/dev/null'.replace(' ', ' '))
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif field_type == "dict" and method in dict_methods:
func_name = dict_methods[method]
self.emit(f'{func_name} "${{this}}_{field}" {args_str} >/dev/null'.replace(' ', ' '))
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif method in str_methods:
func_name = str_methods[method]
self.emit(f'{func_name} "${{__CT_OBJ["$this.{field}"]}}" {args_str} >/dev/null'.replace(' ', ' '))
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
if isinstance(expr.callee.object, ThisExpr) and self.current_class:
method = expr.callee.member
args = [self.generate_expr(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'__ct_class_{self.current_class}_{method} "$this" {args_str} >/dev/null')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif isinstance(expr.callee.object, Identifier):
obj_name = expr.callee.object.name
method = expr.callee.member
args = [self.generate_expr(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
builtin_namespaces = {"fs", "http", "json", "logger", "regex", "args", "shell"}
resolved = self._resolve_qualified(obj_name, method)
if resolved is not None:
if resolved in self.classes:
self.emit(f'{resolved} {args_str}')
self.emit(f'{RET_VAR}="$__ct_last_instance"')
else:
self.emit(f'{resolved} {args_str} >/dev/null')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
if obj_name in self.busing_names:
self.emit_var_assign(RET_VAR, f'$({method} {args_str})')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
if obj_name not in builtin_namespaces:
result = self._dispatch_instance_method_return(obj_name, method, args)
if result:
return True
elif isinstance(expr.callee, Identifier):
func_name = expr.callee.name
resolved = self._resolve_name(func_name)
if resolved in self.classes:
args = [self._generate_call_arg(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'{resolved} {args_str}')
self.emit(f'{RET_VAR}="$__ct_last_instance"')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
if resolved in self.functions:
args = [self._generate_call_arg(arg) for arg in expr.arguments]
args_str = " ".join([f'"{a}"' for a in args])
self.emit(f'{resolved} {args_str} >/dev/null')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
return False
def _dispatch_instance_method_return(self, obj_name: str, method: str, args: list) -> bool:
"""Dispatch instance method for return statement."""
arr_methods = {
"len": "__ct_arr_len", "push": "__ct_arr_push", "pop": "__ct_arr_pop",
"shift": "__ct_arr_shift", "join": "__ct_arr_join", "get": "__ct_arr_get",
"set": "__ct_arr_set", "slice": "__ct_arr_slice",
}
dict_methods = {
"get": "__ct_dict_get", "set": "__ct_dict_set", "has": "__ct_dict_has",
"del": "__ct_dict_del", "keys": "__ct_dict_keys",
}
str_methods = {
"len": "__ct_str_len", "upper": "__ct_str_upper", "lower": "__ct_str_lower",
"trim": "__ct_str_trim", "contains": "__ct_str_contains", "starts": "__ct_str_starts",
"ends": "__ct_str_ends", "index": "__ct_str_index", "replace": "__ct_str_replace",
"substr": "__ct_str_substr", "split": "__ct_str_split", "charAt": "__ct_str_char_at",
"urlencode": "__ct_str_urlencode",
}
args_str = " ".join([f'"{a}"' for a in args])
param_map = getattr(self, 'param_name_map', {})
mapped_name = param_map.get(obj_name, obj_name)
if mapped_name in self.array_vars and method in arr_methods:
func_name = arr_methods[method]
self.emit(f'{func_name} "{mapped_name}" {args_str} >/dev/null'.replace(' ', ' '))
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif mapped_name in self.dict_vars and method in dict_methods:
func_name = dict_methods[method]
self.emit(f'{func_name} "{mapped_name}" {args_str} >/dev/null'.replace(' ', ' '))
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
elif method in str_methods:
func_name = str_methods[method]
self.emit(f'{func_name} "${{{mapped_name}}}" {args_str} >/dev/null'.replace(' ', ' '))
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
self.emit(f'__ct_call_method "${{{mapped_name}}}" "{method}" {args_str} >/dev/null')
self.emit(f'echo "${{{RET_VAR}}}"')
self.emit("return 0")
return True
bootstrap/tokens.py deleted 100644 → 0
View file @ 5f7339cd
from enum import Enum, auto
from dataclasses import dataclass
from typing import Any
class TokenType (Enum):
INTEGER = auto ()
FLOAT = auto ()
STRING = auto ()
TRUE = auto ()
FALSE = auto ()
NIL = auto ()
IDENTIFIER = auto ()
FUNC = auto ()
CLASS = auto ()
CONSTRUCT = auto ()
THIS = auto ()
BASE = auto ()
RETURN = auto ()
IF = auto ()
ELSE = auto ()
FOREACH = auto ()
FOR = auto ()
IN = auto ()
WHILE = auto ()
BREAK = auto ()
CONTINUE = auto ()
TRY = auto ()
EXCEPT = auto ()
FINALLY = auto ()
THROW = auto ()
DEFER = auto ()
RANGE = auto ()
WHEN = auto ()
WITH = auto ()
NEW = auto ()
ASYNC = auto ()
AWAIT = auto ()
ON = auto ()
NAMESPACE = auto ()
USING = auto ()
BUSING = auto ()
PLUS = auto ()
MINUS = auto ()
STAR = auto ()
SLASH = auto ()
PERCENT = auto ()
ASSIGN = auto ()
EQ = auto ()
NEQ = auto ()
LT = auto ()
GT = auto ()
LTE = auto ()
GTE = auto ()
AND = auto ()
OR = auto ()
PIPE = auto ()
NOT = auto ()
ARROW = auto ()
PLUS_ASSIGN = auto ()
MINUS_ASSIGN = auto ()
STAR_ASSIGN = auto ()
SLASH_ASSIGN = auto ()
DOT = auto ()
DOTDOT = auto ()
DOTDOTDOT = auto ()
COLON = auto ()
LPAREN = auto ()
RPAREN = auto ()
LBRACE = auto ()
RBRACE = auto ()
LBRACKET = auto ()
RBRACKET = auto ()
COMMA = auto ()
NEWLINE = auto ()
AT = auto ()
COMMENT = auto ()
EOF = auto ()
KEYWORDS = {
'func': TokenType.FUNC,
'class': TokenType.CLASS,
'construct': TokenType.CONSTRUCT,
'this': TokenType.THIS,
'base': TokenType.BASE,
'return': TokenType.RETURN,
'if': TokenType.IF,
'else': TokenType.ELSE,
'foreach': TokenType.FOREACH,
'for': TokenType.FOR,
'in': TokenType.IN,
'while': TokenType.WHILE,
'break': TokenType.BREAK,
'continue': TokenType.CONTINUE,
'try': TokenType.TRY,
'except': TokenType.EXCEPT,
'finally': TokenType.FINALLY,
'throw': TokenType.THROW,
'defer': TokenType.DEFER,
'range': TokenType.RANGE,
'when': TokenType.WHEN,
'with': TokenType.WITH,
'new': TokenType.NEW,
'async': TokenType.ASYNC,
'await': TokenType.AWAIT,
'on': TokenType.ON,
'namespace': TokenType.NAMESPACE,
'using': TokenType.USING,
'busing': TokenType.BUSING,
'true': TokenType.TRUE,
'false': TokenType.FALSE,
'nil': TokenType.NIL,
}
@dataclass
class Token:
type: TokenType
value: Any
line: int
column: int
def __repr__ (self):
return f"Token({self.type.name}, {self.value!r}, {self.line}:{self.column})"
compiler/__main__.py 0 → 100644
View file @ 31bfe20c
import sys
from .cli import main
sys.exit(main())
compiler/backend/awk/__init__.py 0 → 100644
View file @ 31bfe20c
from .codegen import emit_awk_function
compiler/backend/awk/codegen.py 0 → 100644
View file @ 31bfe20c
from __future__ import annotations
import re
from typing import TYPE_CHECKING
from ...ir.nodes import (
IRFunction, IRBlock, IRAssign, IRFieldAssign, IRIndexAssign,
IRReturn, IRIf, IRWhile, IRFor, IRForeach, IRExprStmt,
IRBreak, IRContinue, IRWhen,
IRInt, IRFloat, IRBool, IRNil, IRString, IRStringInterp, IRStringText,
IRArray, IRDict, IRIdentifier, IRFieldAccess, IRIndexAccess,
IRBinaryOp, IRUnaryOp, IRCall, IRMethodCall, IRLambda,
IRDecorator, IRParam,
)
from ...methods import generate_awk, MATH_METHODS, TIME_METHODS
from ...methods import get_awk_builtin
from ...constants import RET_VAR
if TYPE_CHECKING:
from ..bash.backend import EmitContext
def emit_awk_function(fn: IRFunction, ctx: EmitContext) -> None:
bash_name = fn.symbol.bash_name() if fn.symbol else fn.name
validate_dec = None
for d in fn.decorators:
if d.name == 'validate':
validate_dec = d
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
_body(fn, validate_dec, ctx)
ctx.emit('}')
ctx.emit('')
def _body(fn: IRFunction, validate_dec, ctx: EmitContext) -> None:
var_types = _scan_types(fn.body.stmts if fn.body else [])
stmts = fn.body.stmts if fn.body else []
params_v = [f'-v {p.name}="${{{i+1}}}"' for i, p in enumerate(fn.params)]
params_str = ' '.join(params_v)
awk_cmd = f'"$__ct_awk_cmd" {params_str}'.strip() if params_str else '"$__ct_awk_cmd"'
lines, emit, inc, dec = _emitter()
if validate_dec:
_validation(fn.params, validate_dec, emit)
for s in stmts:
_stmt(s, emit, inc, dec, var_types)
ctx.emit(f"{RET_VAR}=$({awk_cmd} '")
ctx.emit("BEGIN {")
for line in lines:
ctx.emit(f" {_esc(line)}")
ctx.emit("}')")
ctx.emit('local __awk_rc=$?')
ctx.emit(f'echo "${{{RET_VAR}}}"')
ctx.emit('return $__awk_rc')
def _emitter():
lines = []
indent = [0]
def emit(line): lines.append(' ' * indent[0] + line)
def inc(): indent[0] += 1
def dec(): indent[0] -= 1
return lines, emit, inc, dec
def _esc(s: str) -> str:
return s.replace("'", "'\"'\"'")
def _esc_str(s: str) -> str:
s = s.replace('\\', '\\\\')
s = s.replace('\n', '\\n')
s = s.replace('\t', '\\t')
s = s.replace('"', '\\"')
s = s.replace("'", "\\047")
return s
def _scan_types(stmts: list) -> dict:
vt = {}
for s in stmts:
if isinstance(s, IRAssign):
if isinstance(s.value, IRArray):
vt[s.target] = 'array'
elif isinstance(s.value, IRDict):
vt[s.target] = 'dict'
elif isinstance(s.value, IRString):
vt[s.target] = 'string'
elif isinstance(s, IRIf):
vt.update(_scan_types(s.then_block.stmts if s.then_block else []))
for _, b in (s.elif_branches or []):
vt.update(_scan_types(b.stmts if b else []))
if s.else_block:
vt.update(_scan_types(s.else_block.stmts if s.else_block else []))
elif isinstance(s, (IRWhile, IRFor, IRForeach)):
vt.update(_scan_types(s.body.stmts if s.body else []))
return vt
def _stmt(s, emit, inc, dec, vt, in_func=False):
if isinstance(s, IRAssign):
target = s.target
if isinstance(s.value, (IRArray, IRDict)):
if isinstance(s.value, IRArray):
vt[target] = 'array'
else:
vt[target] = 'dict'
emit(f'delete {target}')
return
if _map_filter(s, target, emit, inc, dec, vt):
return
value = _expr(s.value, vt)
op = s.op or '='
if op == '..=':
emit(f'{target} = {target} {value}')
elif op in ('=', '+=', '-=', '*=', '/=', '%='):
emit(f'{target} {op} {value}')
else:
emit(f'{target} = {value}')
elif isinstance(s, IRFieldAssign):
recv = _expr(s.receiver, vt)
emit(f'{recv}["{s.field_name}"] = {_expr(s.value, vt)}')
elif isinstance(s, IRIndexAssign):
obj = _expr(s.object, vt)
idx = _expr(s.index, vt)
emit(f'{obj}[{idx}] = {_expr(s.value, vt)}')
elif isinstance(s, IRReturn):
if in_func:
if s.value:
emit(f'return {_expr(s.value, vt)}')
else:
emit('return')
else:
if s.value:
emit(f'print {_expr(s.value, vt)}')
emit('exit')
elif isinstance(s, IRIf):
cond = _cond(s.condition, vt)
emit(f'if ({cond}) {{')
inc()
for st in (s.then_block.stmts if s.then_block else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
for ec, eb in (s.elif_branches or []):
emit(f'}} else if ({_cond(ec, vt)}) {{')
inc()
for st in (eb.stmts if eb else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
if s.else_block and s.else_block.stmts:
emit('} else {')
inc()
for st in s.else_block.stmts:
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
elif isinstance(s, IRWhile):
emit(f'while ({_cond(s.condition, vt)}) {{')
inc()
for st in (s.body.stmts if s.body else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
elif isinstance(s, IRFor):
var = s.variable
it = s.iterable
if isinstance(it, IRCall) and (it.callee_name or '') in ('range', '__ct_range'):
args = it.args
if len(args) == 1:
start, end, step = '0', _expr(args[0], vt), '1'
elif len(args) == 2:
start, end, step = _expr(args[0], vt), _expr(args[1], vt), '1'
else:
start, end, step = _expr(args[0], vt), _expr(args[1], vt), _expr(args[2], vt)
emit(f'for ({var} = {start}; {var} < {end}; {var} += {step}) {{')
inc()
for st in (s.body.stmts if s.body else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
return
arr = _expr(it, vt)
emit(f'for ({var} in {arr}) {{')
inc()
for st in (s.body.stmts if s.body else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
elif isinstance(s, IRForeach):
vars_ = s.variables
arr = _expr(s.iterable, vt)
if len(vars_) > 1:
emit(f'for ({vars_[0]} in {arr}) {{')
inc()
emit(f'{vars_[1]} = {arr}[{vars_[0]}]')
for st in (s.body.stmts if s.body else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
else:
emit(f'for ({vars_[0]} in {arr}) {{')
inc()
for st in (s.body.stmts if s.body else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
elif isinstance(s, IRExprStmt):
_expr_stmt(s.expr, emit, inc, dec, vt)
elif isinstance(s, IRBreak):
emit('break')
elif isinstance(s, IRContinue):
emit('continue')
elif isinstance(s, IRWhen):
val = _expr(s.value, vt)
emit(f'__when_val = {val}')
first = True
for br in s.branches:
if br.is_else:
emit('} else {')
else:
conds = []
for p in br.patterns:
conds.append(f'__when_val == {_expr(p, vt)}')
for rs, re_ in (br.ranges or []):
conds.append(f'(__when_val >= {_expr(rs, vt)} && __when_val <= {_expr(re_, vt)})')
cond_str = ' || '.join(conds)
if first:
emit(f'if ({cond_str}) {{')
first = False
else:
emit(f'}} else if ({cond_str}) {{')
inc()
for st in (br.body.stmts if br.body else []):
_stmt(st, emit, inc, dec, vt, in_func)
dec()
emit('}')
elif isinstance(s, IRBlock):
for st in (s.stmts if s else []):
_stmt(st, emit, inc, dec, vt, in_func)
def _expr_stmt(expr, emit, inc, dec, vt):
if isinstance(expr, IRCall):
name = expr.callee_name or ''
if name in ('assert', '__ct_assert'):
cond = _cond(expr.args[0], vt) if expr.args else '1'
msg = _expr(expr.args[1], vt) if len(expr.args) >= 2 else '"Assertion failed"'
emit(f'if (!({cond})) {{ print {msg} > "/dev/stderr"; exit 1 }}')
return
if name in ('assert_eq', '__ct_assert_eq'):
expected = _expr(expr.args[0], vt) if expr.args else '""'
actual = _expr(expr.args[1], vt) if len(expr.args) >= 2 else '""'
msg = _expr(expr.args[2], vt) if len(expr.args) >= 3 else '"Values not equal"'
emit(f'if ({expected} != {actual}) {{ print {msg} > "/dev/stderr"; exit 1 }}')
return
if name in ('print', '__ct_print'):
awk_args = [_expr(a, vt) for a in expr.args]
emit(f"print {', '.join(awk_args)}" if awk_args else 'print')
return
if isinstance(expr, IRMethodCall):
if isinstance(expr.receiver, IRIdentifier):
obj = expr.receiver.name
method = expr.method_name
awk_args = [_expr(a, vt) for a in expr.args]
var_type = vt.get(obj, 'string')
type_name = {'array': 'array', 'dict': 'dict'}.get(var_type, 'string')
code = generate_awk(type_name, method, obj, awk_args)
if code:
emit(code)
return
code = _expr(expr, vt)
if code:
emit(code)
def _expr(expr, vt) -> str:
if expr is None:
return ''
if isinstance(expr, IRInt):
return str(expr.value)
if isinstance(expr, IRFloat):
return str(expr.value)
if isinstance(expr, IRString):
parts = expr.parts
has_interp = any(isinstance(p, IRStringInterp) for p in parts)
if has_interp:
return _interp(parts, vt)
text = ''.join(p.text for p in parts if isinstance(p, IRStringText))
return f'"{_esc_str(text)}"'
if isinstance(expr, IRBool):
return '1' if expr.value else '0'
if isinstance(expr, IRNil):
return '""'
if isinstance(expr, IRIdentifier):
if expr.name.startswith('env.'):
return f'ENVIRON["{expr.name[4:]}"]'
return expr.name
if isinstance(expr, (IRArray, IRDict)):
return '""'
if isinstance(expr, IRBinaryOp):
left = _expr(expr.left, vt)
right = _expr(expr.right, vt)
op = expr.operator
if op == '..':
return f'({left} {right})'
return f'({left} {op} {right})'
if isinstance(expr, IRUnaryOp):
operand = _expr(expr.operand, vt)
if expr.operator == '!':
return f'(!{operand})'
if expr.operator == '-':
return f'(-{operand})'
return operand
if isinstance(expr, IRCall):
name = expr.callee_name or ''
args = [_expr(a, vt) for a in expr.args]
code = get_awk_builtin(name, args)
if code:
return code
return f"{name}({', '.join(args)})"
if isinstance(expr, IRMethodCall):
if isinstance(expr.receiver, IRIdentifier):
obj = expr.receiver.name
method = expr.method_name
args = [_expr(a, vt) for a in expr.args]
if obj == 'math' and method in MATH_METHODS:
m = MATH_METHODS[method]
if m.awk_builtin:
return m.awk_builtin(args)
if obj == 'time' and method in TIME_METHODS:
m = TIME_METHODS[method]
if m.awk_builtin:
return m.awk_builtin(args)
var_type = vt.get(obj, 'string')
type_name = {'array': 'array', 'dict': 'dict'}.get(var_type, 'string')
code = generate_awk(type_name, method, obj, args)
if code:
return code
return ''
if isinstance(expr, IRIndexAccess):
obj = _expr(expr.object, vt)
idx = _expr(expr.index, vt)
return f'{obj}[{idx}]'
if isinstance(expr, IRFieldAccess):
if isinstance(expr.receiver, IRIdentifier) and expr.receiver.name == 'env':
return f'ENVIRON["{expr.field_name}"]'
recv = _expr(expr.receiver, vt)
return f'{recv}["{expr.field_name}"]'
return ''
def _cond(expr, vt) -> str:
result = _expr(expr, vt)
if result.startswith('(') and result.endswith(')'):
return result[1:-1]
return result
def _interp(parts: list, vt) -> str:
result = []
for p in parts:
if isinstance(p, IRStringText):
if p.text:
result.append(f'"{_esc_str(p.text)}"')
elif isinstance(p, IRStringInterp):
result.append(_expr(p.expr, vt))
if len(result) == 1:
return result[0]
return '(' + ' '.join(result) + ')'
def _map_filter(s: IRAssign, target: str, emit, inc, dec, vt) -> bool:
if not isinstance(s.value, IRMethodCall):
return False
if not isinstance(s.value.receiver, IRIdentifier):
return False
method = s.value.method_name
if method not in ('map', 'filter'):
return False
args = s.value.args
if not args or not isinstance(args[0], IRLambda):
return False
arr = s.value.receiver.name
lam = args[0]
param = lam.params[0] if lam.params else '__x'
if hasattr(param, 'name'):
param = param.name
body_expr = _lambda_body(lam, vt)
vt[target] = 'array'
emit(f'delete {target}')
if method == 'map':
emit(f'__ct_len = length({arr})')
emit(f'for (__i = 1; __i <= __ct_len; __i++) {{')
inc()
emit(f'{param} = {arr}[__i]')
emit(f'{target}[__i] = {body_expr}')
dec()
emit('}')
else:
emit(f'__ct_len = length({arr})')
emit(f'__ct_fidx = 0')
emit(f'for (__i = 1; __i <= __ct_len; __i++) {{')
inc()
emit(f'{param} = {arr}[__i]')
emit(f'if ({body_expr}) {{')
inc()
emit(f'__ct_fidx++')
emit(f'{target}[__ct_fidx] = {arr}[__i]')
dec()
emit('}')
dec()
emit('}')
return True
def _lambda_body(lam: IRLambda, vt) -> str:
if lam.body:
stmts = lam.body.stmts if isinstance(lam.body, IRBlock) else [lam.body]
for s in stmts:
if isinstance(s, IRReturn) and s.value:
return _expr(s.value, vt)
if lam.body_expr:
return _expr(lam.body_expr, vt)
return '""'
def _validation(params: list, dec: IRDecorator, emit) -> None:
validations = {}
for aname, aval in dec.args:
if aname and isinstance(aval, IRString):
text = ''.join(p.text for p in aval.parts if isinstance(p, IRStringText))
validations[aname] = text
for p in params:
rule = validations.get(p.name)
if not rule:
continue
pname = p.name
if 'int' in rule:
emit(f'if ({pname} !~ /^-?[0-9]+$/) {{ print "{pname} must be integer" > "/dev/stderr"; exit 1 }}')
for op, val in re.findall(r'(>|<|>=|<=|==|!=)\s*(-?\d+)', rule):
emit(f'if (!({pname} {op} {val})) {{ print "{pname} must be {op} {val}" > "/dev/stderr"; exit 1 }}')
compiler/backend/bash/__init__.py 0 → 100644
View file @ 31bfe20c
from .backend import BashBackend, compile_to_bash
__all__ = ['BashBackend', 'compile_to_bash']
compiler/backend/bash/awk.py 0 → 100644
View file @ 31bfe20c
"""
AWK code generator for @awk decorated functions.
Converts IR nodes into AWK program text embedded in a bash wrapper.
"""
from __future__ import annotations
import re
from typing import TYPE_CHECKING
from ...ir.nodes import (
IRFunction, IRBlock, IRAssign, IRReturn, IRIf, IRWhile, IRFor,
IRForeach, IRExprStmt, IRBreak, IRContinue, IRWhen,
IRInt, IRFloat, IRBool, IRNil, IRString, IRStringInterp, IRStringText,
IRArray, IRDict, IRIdentifier, IRFieldAccess, IRIndexAccess,
IRBinaryOp, IRUnaryOp, IRCall, IRMethodCall, IRLambda,
IRExpr, IRStmt, IRDecorator, IRParam,
)
from ...methods import generate_awk, MATH_METHODS, TIME_METHODS
from ...methods import get_awk_builtin
from ...constants import RET_VAR
if TYPE_CHECKING:
from .backend import EmitContext
def emit_awk_function(fn: IRFunction, ctx: EmitContext) -> None:
bash_name = fn.symbol.bash_name() if fn.symbol else fn.name
validate_dec = None
for d in fn.decorators:
if d.name == 'validate':
validate_dec = d
ctx.emit(f'{bash_name} () {{')
ctx.indent()
_emit_awk_body(fn, validate_dec, ctx)
ctx.dedent()
ctx.emit('}')
ctx.emit('')
def _emit_awk_body(fn: IRFunction, validate_dec, ctx: EmitContext) -> None:
var_types = _scan_types(fn.body.stmts if fn.body else [])
nested_funcs = []
main_stmts = []
for s in (fn.body.stmts if fn.body else []):
if isinstance(s, IRExprStmt) and isinstance(s.expr, IRCall) and not s.expr.is_shell_cmd:
main_stmts.append(s)
elif isinstance(s, IRFunction):
nested_funcs.append(s)
else:
main_stmts.append(s)
params_v = [f'-v {p.name}="${{{i+1}}}"' for i, p in enumerate(fn.params)]
params_str = " ".join(params_v)
awk_cmd = f'"$__ct_awk_cmd" {params_str}'.strip() if params_str else '"$__ct_awk_cmd"'
lines, emit, inc, dec = _make_emitter()
if validate_dec:
_emit_validation(fn.params, validate_dec, emit)
for stmt in main_stmts:
_awk_stmt(stmt, emit, inc, dec, var_types)
ctx.emit(f"{RET_VAR}=$({awk_cmd} '")
for nf in nested_funcs:
_emit_helper_func(nf, ctx, var_types)
ctx.emit("BEGIN {")
for line in lines:
ctx.emit(f" {_awk_escape(line)}")
ctx.emit("}')")
ctx.emit('local __awk_rc=$?')
ctx.emit(f'echo "${{{RET_VAR}}}"')
ctx.emit('return $__awk_rc')
def _make_emitter():
lines = []
indent = [0]
def emit(line): lines.append(" " * indent[0] + line)
def inc(): indent[0] += 1
def dec_(): indent[0] -= 1
return lines, emit, inc, dec_
def _awk_escape(s: str) -> str:
return s.replace("'", "'\"'\"'")
def _awk_escape_str(s: str) -> str:
s = s.replace('\\', '\\\\')
s = s.replace('\n', '\\n')
s = s.replace('\t', '\\t')
s = s.replace('"', '\\"')
s = s.replace("'", "\\047")
return s
def _scan_types(stmts: list) -> dict:
var_types = {}
for stmt in stmts:
if isinstance(stmt, IRAssign):
name = stmt.target
if isinstance(stmt.value, IRArray):
var_types[name] = "array"
elif isinstance(stmt.value, IRDict):
var_types[name] = "dict"
elif isinstance(stmt.value, IRString):
var_types[name] = "string"
elif isinstance(stmt, IRIf):
var_types.update(_scan_types(stmt.then_block.stmts if stmt.then_block else []))
for _, b in (stmt.elif_branches or []):
var_types.update(_scan_types(b.stmts if b else []))
if stmt.else_block:
var_types.update(_scan_types(stmt.else_block.stmts if stmt.else_block else []))
elif isinstance(stmt, (IRWhile, IRFor, IRForeach)):
var_types.update(_scan_types(stmt.body.stmts if stmt.body else []))
return var_types
# ---------------------------------------------------------------------------
# Statements
# ---------------------------------------------------------------------------
def _awk_stmt(stmt, emit, inc, dec, var_types, in_func=False):
if isinstance(stmt, IRAssign):
target = stmt.target
if isinstance(stmt.value, (IRArray, IRDict)):
if isinstance(stmt.value, IRArray):
var_types[target] = "array"
else:
var_types[target] = "dict"
emit(f"delete {target}")
return
if _handle_map_filter(stmt, target, emit, inc, dec, var_types):
return
value = _awk_expr(stmt.value, var_types)
op = getattr(stmt, 'op', '=') or '='
if op == '..=':
emit(f"{target} = {target} {value}")
elif op in ('=', '+=', '-=', '*=', '/=', '%='):
emit(f"{target} {op} {value}")
else:
emit(f"{target} = {value}")
elif isinstance(stmt, IRReturn):
if in_func:
if stmt.value:
emit(f"return {_awk_expr(stmt.value, var_types)}")
else:
emit("return")
else:
if stmt.value:
emit(f"print {_awk_expr(stmt.value, var_types)}")
emit("exit")
elif isinstance(stmt, IRIf):
cond = _awk_cond(stmt.condition, var_types)
emit(f"if ({cond}) {{")
inc()
for s in (stmt.then_block.stmts if stmt.then_block else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
for elif_cond, elif_block in (stmt.elif_branches or []):
emit(f"}} else if ({_awk_cond(elif_cond, var_types)}) {{")
inc()
for s in (elif_block.stmts if elif_block else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
if stmt.else_block and stmt.else_block.stmts:
emit("} else {")
inc()
for s in stmt.else_block.stmts:
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
elif isinstance(stmt, IRWhile):
emit(f"while ({_awk_cond(stmt.condition, var_types)}) {{")
inc()
for s in (stmt.body.stmts if stmt.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
elif isinstance(stmt, IRFor):
var = stmt.variable
if isinstance(stmt.iterable, IRCall) and stmt.iterable.callee_name in ('range', '__ct_range'):
args = stmt.iterable.args
if len(args) == 1:
start, end, step = "0", _awk_expr(args[0], var_types), "1"
elif len(args) == 2:
start = _awk_expr(args[0], var_types)
end = _awk_expr(args[1], var_types)
step = "1"
else:
start = _awk_expr(args[0], var_types)
end = _awk_expr(args[1], var_types)
step = _awk_expr(args[2], var_types)
emit(f"for ({var} = {start}; {var} < {end}; {var} += {step}) {{")
inc()
for s in (stmt.body.stmts if stmt.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
return
arr = _awk_expr(stmt.iterable, var_types)
emit(f"for ({var} in {arr}) {{")
inc()
for s in (stmt.body.stmts if stmt.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
elif isinstance(stmt, IRForeach):
vars_ = stmt.variables
iterable = stmt.iterable
arr = _awk_expr(iterable, var_types)
if len(vars_) > 1:
emit(f"for ({vars_[0]} in {arr}) {{")
inc()
emit(f"{vars_[1]} = {arr}[{vars_[0]}]")
for s in (stmt.body.stmts if stmt.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
else:
emit(f"for ({vars_[0]} in {arr}) {{")
inc()
for s in (stmt.body.stmts if stmt.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
elif isinstance(stmt, IRExprStmt):
_awk_expr_stmt(stmt.expr, emit, inc, dec, var_types)
elif isinstance(stmt, IRBreak):
emit("break")
elif isinstance(stmt, IRContinue):
emit("continue")
elif isinstance(stmt, IRWhen):
val = _awk_expr(stmt.value, var_types)
emit(f"__when_val = {val}")
first = True
for branch in stmt.branches:
if branch.is_else:
emit("} else {")
else:
conditions = []
for p in branch.patterns:
if hasattr(p, 'start') and hasattr(p, 'end'):
s = _awk_expr(p.start, var_types)
e = _awk_expr(p.end, var_types)
conditions.append(f"(__when_val >= {s} && __when_val <= {e})")
else:
pval = _awk_expr(p, var_types)
conditions.append(f"__when_val == {pval}")
cond_str = " || ".join(conditions)
if first:
emit(f"if ({cond_str}) {{")
first = False
else:
emit(f"}} else if ({cond_str}) {{")
inc()
for s in (branch.body.stmts if branch.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
dec()
emit("}")
elif isinstance(stmt, IRBlock):
for s in (stmt.stmts if stmt else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func)
def _awk_expr_stmt(expr, emit, inc, dec, var_types):
if isinstance(expr, IRCall):
callee = expr.callee_name or ''
args_list = expr.args
if callee in ('assert', '__ct_assert'):
cond = _awk_cond(args_list[0], var_types) if args_list else "1"
msg = _awk_expr(args_list[1], var_types) if len(args_list) >= 2 else '"Assertion failed"'
emit(f'if (!({cond})) {{ print {msg} > "/dev/stderr"; exit 1 }}')
return
if callee in ('assert_eq', '__ct_assert_eq'):
expected = _awk_expr(args_list[0], var_types) if args_list else '""'
actual = _awk_expr(args_list[1], var_types) if len(args_list) >= 2 else '""'
msg = _awk_expr(args_list[2], var_types) if len(args_list) >= 3 else '"Values not equal"'
emit(f'if ({expected} != {actual}) {{ print {msg} > "/dev/stderr"; exit 1 }}')
return
if callee in ('print', '__ct_print'):
awk_args = [_awk_expr(a, var_types) for a in args_list]
emit(f"print {', '.join(awk_args)}" if awk_args else "print")
return
if isinstance(expr, IRMethodCall):
if isinstance(expr.receiver, IRIdentifier):
obj_name = expr.receiver.name
method = expr.method_name
awk_args = [_awk_expr(a, var_types) for a in expr.args]
var_type = var_types.get(obj_name, "string")
type_name = {"array": "array", "dict": "dict"}.get(var_type, "string")
awk_code = generate_awk(type_name, method, obj_name, awk_args)
if awk_code:
emit(awk_code)
return
code = _awk_expr(expr, var_types)
if code:
emit(code)
# ---------------------------------------------------------------------------
# Expressions
# ---------------------------------------------------------------------------
def _awk_expr(expr, var_types) -> str:
if expr is None:
return ""
if isinstance(expr, IRInt):
return str(expr.value)
if isinstance(expr, IRFloat):
return str(expr.value)
if isinstance(expr, IRString):
parts = expr.parts
has_interp = any(isinstance(p, IRStringInterp) for p in parts)
if has_interp:
return _awk_interp_parts(parts, var_types)
text = ''.join(p.value for p in parts if isinstance(p, IRStringText))
return f'"{_awk_escape_str(text)}"'
if isinstance(expr, IRBool):
return "1" if expr.value else "0"
if isinstance(expr, IRNil):
return '""'
if isinstance(expr, IRIdentifier):
if expr.name.startswith('env.'):
return f'ENVIRON["{expr.name[4:]}"]'
return expr.name
if isinstance(expr, IRArray):
return '""'
if isinstance(expr, IRDict):
return '""'
if isinstance(expr, IRBinaryOp):
left = _awk_expr(expr.left, var_types)
right = _awk_expr(expr.right, var_types)
op = expr.op
if op == "..":
return f"({left} {right})"
return f"({left} {op} {right})"
if isinstance(expr, IRUnaryOp):
operand = _awk_expr(expr.operand, var_types)
if expr.op == "!":
return f"(!{operand})"
if expr.op == "-":
return f"(-{operand})"
return operand
if isinstance(expr, IRCall):
callee = expr.callee_name or ''
args = [_awk_expr(a, var_types) for a in expr.args]
awk_code = get_awk_builtin(callee, args)
if awk_code:
return awk_code
return f"{callee}({', '.join(args)})"
if isinstance(expr, IRMethodCall):
if isinstance(expr.receiver, IRIdentifier):
obj_name = expr.receiver.name
method = expr.method_name
args = [_awk_expr(a, var_types) for a in expr.args]
if obj_name == 'math' and method in MATH_METHODS:
m = MATH_METHODS[method]
if m.awk_builtin:
return m.awk_builtin(args)
if obj_name == 'time' and method in TIME_METHODS:
m = TIME_METHODS[method]
if m.awk_builtin:
return m.awk_builtin(args)
var_type = var_types.get(obj_name, "string")
type_name = {"array": "array", "dict": "dict"}.get(var_type, "string")
awk_code = generate_awk(type_name, method, obj_name, args)
if awk_code:
return awk_code
return ""
if isinstance(expr, IRIndexAccess):
obj = _awk_expr(expr.obj, var_types)
idx = _awk_expr(expr.index, var_types)
return f"{obj}[{idx}]"
if isinstance(expr, IRFieldAccess):
if isinstance(expr.receiver, IRIdentifier) and expr.receiver.name == 'env':
return f'ENVIRON["{expr.field_name}"]'
recv = _awk_expr(expr.receiver, var_types)
return f'{recv}["{expr.field_name}"]'
return ""
def _awk_cond(expr, var_types) -> str:
result = _awk_expr(expr, var_types)
if result.startswith('(') and result.endswith(')'):
return result[1:-1]
return result
def _awk_interp_parts(parts: list, var_types) -> str:
result_parts = []
for p in parts:
if isinstance(p, IRStringText):
if p.value:
result_parts.append(f'"{_awk_escape_str(p.value)}"')
elif isinstance(p, IRStringInterp):
result_parts.append(_awk_expr(p.expr, var_types))
if len(result_parts) == 1:
return result_parts[0]
return '(' + ' '.join(result_parts) + ')'
# ---------------------------------------------------------------------------
# map/filter
# ---------------------------------------------------------------------------
def _handle_map_filter(stmt: IRAssign, target: str, emit, inc, dec, var_types) -> bool:
if not isinstance(stmt.value, IRMethodCall):
return False
if not isinstance(stmt.value.receiver, IRIdentifier):
return False
method = stmt.value.method_name
if method not in ('map', 'filter'):
return False
args = stmt.value.args
if not args or not isinstance(args[0], IRLambda):
return False
arr_name = stmt.value.receiver.name
lam = args[0]
param = lam.params[0].name if lam.params else "__x"
body_expr = _lambda_body_expr(lam, var_types)
var_types[target] = "array"
emit(f"delete {target}")
if method == "map":
emit(f"__ct_len = length({arr_name})")
emit(f"for (__i = 1; __i <= __ct_len; __i++) {{")
inc()
emit(f"{param} = {arr_name}[__i]")
emit(f"{target}[__i] = {body_expr}")
dec()
emit("}")
elif method == "filter":
emit(f"__ct_len = length({arr_name})")
emit(f"__ct_fidx = 0")
emit(f"for (__i = 1; __i <= __ct_len; __i++) {{")
inc()
emit(f"{param} = {arr_name}[__i]")
emit(f"if ({body_expr}) {{")
inc()
emit(f"__ct_fidx++")
emit(f"{target}[__ct_fidx] = {arr_name}[__i]")
dec()
emit("}")
dec()
emit("}")
return True
def _lambda_body_expr(lam: IRLambda, var_types) -> str:
if lam.body and lam.body.stmts:
for s in lam.body.stmts:
if isinstance(s, IRReturn) and s.value:
return _awk_expr(s.value, var_types)
if lam.expr:
return _awk_expr(lam.expr, var_types)
return '""'
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _emit_helper_func(nf: IRFunction, ctx: EmitContext, var_types: dict) -> None:
params = ", ".join(p.name for p in nf.params)
ctx.emit(f"function {nf.name}({params}) {{")
lines, emit, inc, dec = _make_emitter()
for s in (nf.body.stmts if nf.body else []):
_awk_stmt(s, emit, inc, dec, var_types, in_func=True)
for line in lines:
ctx.emit(f" {_awk_escape(line)}")
ctx.emit("}")
def _emit_validation(params: list[IRParam], dec: IRDecorator, emit) -> None:
validations = {}
for aname, aval in dec.args:
if aname and isinstance(aval, IRString):
text = ''.join(p.value for p in aval.parts if isinstance(p, IRStringText))
validations[aname] = text
for p in params:
rule = validations.get(p.name)
if not rule:
continue
pname = p.name
if "int" in rule:
emit(f'if ({pname} !~ /^-?[0-9]+$/) {{ print "{pname} must be integer" > "/dev/stderr"; exit 1 }}')
for op, val in re.findall(r'(>|<|>=|<=|==|!=)\s*(-?\d+)', rule):
emit(f'if (!({pname} {op} {val})) {{ print "{pname} must be {op} {val}" > "/dev/stderr"; exit 1 }}')
compiler/backend/bash/backend.py 0 → 100644
View file @ 31bfe20c
"""
BashBackend — main coordinator.
Usage:
from compiler.backend.bash.backend import BashBackend
backend = BashBackend()
bash_code = backend.emit(ir, used_stdlib_categories)
"""
from __future__ import annotations
import contextlib
from typing import Iterator
from ...ir.nodes import (
IRProgram, IRFunction, IRClass, IRBlock, IRStmt, IRExpr,
IRLambda, IRParam,
)
from ...semantics.scope import Symbol
from .constants import RET_VAR, LAMBDA_PREFIX, COPROC_PREFIX, TMP_PREFIX
from .stdlib import emit_stdlib
from .expr import emit_expr, emit_expr_as_stmt
from .stmt import emit_stmt, emit_block
from .classes import emit_class, emit_function
# ---------------------------------------------------------------------------
# EmitContext
# ---------------------------------------------------------------------------
class EmitContext:
"""
Carries state needed during code generation:
- current output buffer
- indentation level
- in_function flag + local_vars set
- counters for fresh names
"""
def __init__(
self,
output: list[str],
indent_level: int = 0,
lambda_defs: list[str] | None = None,
in_function: bool = False,
array_vars: 'set[str] | None' = None,
dict_vars: 'set[str] | None' = None,
process_handles: 'dict[str, str] | None' = None,
) -> None:
self._output = output
self._indent = indent_level
self._lambda_defs = lambda_defs if lambda_defs is not None else []
self._locals: set[str] = set()
self._tmp_counter = 0
self._coproc_counter = 0
self._lambda_counter = 0
self.in_function = in_function
self.array_vars: set[str] = array_vars if array_vars is not None else set()
self.dict_vars: set[str] = dict_vars if dict_vars is not None else set()
# var_name → coproc_name (e.g. "proc" → "__ct_cp1")
self.process_handles: dict[str, str] = process_handles if process_handles is not None else {}
# param_array/dict_vars: function params holding an array/dict name (pass "${param}")
self.param_array_vars: set[str] = set()
self.param_dict_vars: set[str] = set()
# ------------------------------------------------------------------ emit
def emit(self, line: str = '') -> None:
if line:
self._output.append(' ' * self._indent + line)
else:
self._output.append('')
@contextlib.contextmanager
def indented(self) -> Iterator[None]:
self._indent += 1
try:
yield
finally:
self._indent -= 1
# ------------------------------------------------------------------ names
def fresh_tmp(self) -> str:
self._tmp_counter += 1
return f'{TMP_PREFIX}{self._tmp_counter}'
def emit_tmp(self, name: str, val: str) -> None:
"""Emit tmp variable assignment — local inside functions, global outside."""
if self.in_function:
self.emit(f'local {name}={val}')
else:
self.emit(f'{name}={val}')
def fresh_coproc(self) -> str:
self._coproc_counter += 1
return f'{COPROC_PREFIX}{self._coproc_counter}'
# ------------------------------------------------------------------ locals
def declare_local(self, name: str) -> None:
self._locals.add(name)
def is_declared(self, name: str) -> bool:
return name in self._locals
# ------------------------------------------------------------------ lambda
def emit_lambda(self, node: IRLambda) -> str:
"""Emit a lambda as a named bash function; return its name."""
self._lambda_counter += 1
name = f'{LAMBDA_PREFIX}{self._lambda_counter}'
# Build lambda body in a separate context
lam_lines: list[str] = []
lam_ctx = EmitContext(lam_lines, indent_level=1, lambda_defs=self._lambda_defs)
lam_lines.append(f'{name} () {{')
for i, p in enumerate(node.params):
lam_lines.append(f' local {p}="${{{i + 1}}}"')
if node.body_expr is not None:
val = emit_expr(node.body_expr, lam_ctx)
lam_lines.append(f' {RET_VAR}={val}')
elif node.body is not None:
emit_block(node.body, lam_ctx)
lam_lines.append('}')
self._lambda_defs.extend(lam_lines)
return name
# ------------------------------------------------------------------ child contexts
def function_context(self) -> '_FunctionContext':
return _FunctionContext(self._indent, self._lambda_defs)
def child_context(self, output: list[str]) -> 'EmitContext':
child = EmitContext(output, self._indent, self._lambda_defs, self.in_function,
self.array_vars, self.dict_vars, self.process_handles)
return child
def lines(self) -> list[str]:
return self._output
class _FunctionContext(EmitContext):
"""Context inside a function body — tracks local vars."""
def __init__(self, indent: int, lambda_defs: list[str],
array_vars: 'set[str] | None' = None,
dict_vars: 'set[str] | None' = None) -> None:
super().__init__([], indent, lambda_defs, in_function=True, array_vars=array_vars, dict_vars=dict_vars)
# ---------------------------------------------------------------------------
# BashBackend
# ---------------------------------------------------------------------------
class BashBackend:
"""Emits a complete bash script from an IRProgram."""
def emit(self, ir: IRProgram, used_categories: set[str]) -> str:
lines: list[str] = []
lambda_defs: list[str] = []
# Shebang + safety flags
lines.append('#!/usr/bin/env bash')
lines.append('set -euo pipefail')
lines.append('')
# busing imports
for bus in getattr(ir, 'busing', []):
lines.append(f'source "{bus.path}"')
if getattr(ir, 'busing', []):
lines.append('')
# Standard library
emit_stdlib(lines, used_categories)
# Lambda definitions collected during codegen
top_ctx = EmitContext(lines, lambda_defs=lambda_defs)
# DCE comments for skipped @test functions
for name in getattr(ir, 'skipped_tests', []):
lines.append(f'# DCE: skipped @test function {name}')
if getattr(ir, 'skipped_tests', []):
lines.append('')
# Classes
all_classes = {cls.name: cls for cls in ir.classes}
for cls in ir.classes:
emit_class(cls, top_ctx, all_classes)
# Functions (non-method, non-awk)
for fn in ir.functions:
if not fn.is_method:
emit_function(fn, top_ctx)
# Lambda defs from function bodies — emit before top-level stmts
fn_lambda_defs = list(lambda_defs)
lambda_defs.clear()
# Top-level statements into a temporary buffer
top_stmt_lines: list[str] = []
if ir.top_stmts:
stmt_ctx = EmitContext(top_stmt_lines, lambda_defs=lambda_defs)
for stmt in ir.top_stmts:
emit_stmt(stmt, stmt_ctx)
# Now emit: function-body lambdas, then top-stmt lambdas, then top stmts
all_lambdas = fn_lambda_defs + lambda_defs
if all_lambdas:
lines.append('# === Lambdas ===')
lines.extend(all_lambdas)
lines.append('')
if top_stmt_lines:
lines.extend(top_stmt_lines)
lines.append('')
# Call main() if defined (and no top-level stmts already called it)
has_main = any(fn.name == 'main' and not fn.is_method for fn in ir.functions)
if has_main:
lines.append('main "$@"')
# Test runner: call @test functions and print summary
test_fns = [fn for fn in ir.functions if fn.is_test and not fn.is_method]
if test_fns:
lines.append('')
lines.append('# === Test Runner ===')
for fn in test_fns:
desc = ''
for d in fn.decorators:
if d.name == 'test' and d.args:
from ..bash.expr import emit_expr
desc_ctx = EmitContext([], lambda_defs=[])
desc = emit_expr(d.args[0][1], desc_ctx)
break
bash_name = fn.symbol.bash_name() if fn.symbol else fn.name
if desc:
lines.append(f'__ct_test_current={desc}')
else:
lines.append(f'__ct_test_current="{bash_name}"')
lines.append(f'echo -n " $__ct_test_current ... "')
lines.append(f'__ct_before=$__ct_test_failed')
lines.append(f'{bash_name} || __ct_test_failed=$((__ct_test_failed+1))')
lines.append(f'[[ $__ct_test_failed -eq $__ct_before ]] && echo "PASS" || echo "FAIL"')
lines.append('echo ""')
lines.append('echo "$__ct_test_passed tests passed, $__ct_test_failed failed"')
lines.append('[[ $__ct_test_failed -eq 0 ]] || exit 1')
return '\n'.join(lines)
# ---------------------------------------------------------------------------
# Public convenience function
# ---------------------------------------------------------------------------
def compile_to_bash(ir: IRProgram, used_categories: set[str]) -> str:
"""Convenience wrapper."""
return BashBackend().emit(ir, used_categories)
compiler/backend/bash/classes.py 0 → 100644
View file @ 31bfe20c
"""
Class and method code generation for BashBackend.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from ...ir.nodes import (
IRClass, IRFunction, IRClassField, IRExpr, IRNil, IRArray, IRDict, IRInt,
IRMethodCall, IRIdentifier, IRForeach, IRDecorator,
)
from ...semantics.types import T_ANY
from .constants import OBJ_STORE, RET_VAR, CLASS_FUNC_PREFIX
from .expr import _expr, emit_expr
from .stmt import emit_block
from ..awk import emit_awk_function
if TYPE_CHECKING:
from .backend import EmitContext
def emit_class(cls: IRClass, ctx: 'EmitContext',
all_classes: 'dict[str, IRClass] | None' = None) -> None:
"""Emit all bash code for a class."""
_emit_constructor_func(cls, ctx)
_emit_class_metadata(cls, ctx)
if cls.constructor:
_emit_construct_method(cls, ctx)
for method in cls.methods:
if method.is_awk:
ctx.emit(f'# @awk method {cls.name}.{method.name} — see AWK section')
else:
_emit_method(cls, method, ctx)
# Inheritance: emit parent method proxy functions
if cls.parent and all_classes:
own_names = {m.name for m in cls.methods}
if cls.constructor:
own_names.add('construct')
parent_cls = all_classes.get(cls.parent)
if parent_cls:
for pm in parent_cls.methods:
if pm.name not in own_names and not pm.is_awk:
child_fn = f'__ct_class_{cls.name}_{pm.name}'
parent_fn = f'__ct_class_{cls.parent}_{pm.name}'
ctx.emit(f'{child_fn} () {{ {parent_fn} "$@"; }}')
ctx.emit('')
def _emit_constructor_func(cls: IRClass, ctx: 'EmitContext') -> None:
"""Emit the bash constructor function: ClassName () { ... }"""
name = cls.symbol.bash_name() if cls.symbol else cls.name
ctx.emit(f'{name} () {{')
with ctx.indented():
# Generate unique instance ID
ctx.emit(f'local __ct_id="{cls.name}_$$_$RANDOM"')
ctx.emit(f'__ct_obj_class["${{__ct_id}}"]={cls.name!r}')
ctx.emit(f'__ct_last_instance="$__ct_id"')
# Initialise fields
for f in cls.fields:
_init_field(cls.name, f, ctx)
# Call user-defined construct if present, passing all args
if cls.constructor:
ctx.emit(f'{CLASS_FUNC_PREFIX}{cls.name}_construct "${{__ct_id}}" "$@"')
ctx.emit('}')
ctx.emit('')
def _init_field(cls_name: str, field: IRClassField, ctx: 'EmitContext') -> None:
fname = field.name
kind = field.field_kind
defval = field.default
if kind == 'array':
# Variable name is "${__ct_id}_fname" — unique per instance, no $$ in literal name
if defval and isinstance(defval, IRArray):
elems = ' '.join(_expr(e, ctx) for e in defval.elements)
ctx.emit(f'declare -ga "${{__ct_id}}_{fname}=({elems})"')
else:
ctx.emit(f'declare -ga "${{__ct_id}}_{fname}=()"')
# Store the array variable name (which expands at runtime to the per-instance name)
ctx.emit(f'{OBJ_STORE}["${{__ct_id}}.{fname}"]="${{__ct_id}}_{fname}"')
elif kind == 'dict':
# Variable name is "${__ct_id}_fname_d" — unique per instance
if defval and isinstance(defval, IRDict):
pairs = ' '.join(
f'[{_expr(k, ctx)}]={_expr(v, ctx)}'
for k, v in defval.pairs
)
ctx.emit(f'declare -gA "${{__ct_id}}_{fname}_d=({pairs})"')
else:
ctx.emit(f'declare -gA "${{__ct_id}}_{fname}_d=()"')
ctx.emit(f'{OBJ_STORE}["${{__ct_id}}.{fname}"]="${{__ct_id}}_{fname}_d"')
else:
if defval and not isinstance(defval, IRNil):
val = _expr(defval, ctx)
else:
val = '""'
ctx.emit(f'{OBJ_STORE}["${{__ct_id}}.{fname}"]={val}')
def _emit_class_metadata(cls: IRClass, ctx: 'EmitContext') -> None:
"""Emit metadata arrays for reflect + json.unmarshal."""
name = cls.name
field_names = [f.name for f in cls.fields]
field_types = {f.name: f.type.class_name or f.type.kind for f in cls.fields}
fields_str = ' '.join(f'"{fn}"' for fn in field_names)
ctx.emit(f'declare -ga __ct_class_meta_{name}_fields=({fields_str})')
type_pairs = ' '.join(f'[{fn!r}]={ft!r}' for fn, ft in field_types.items())
ctx.emit(f'declare -gA __ct_class_meta_{name}_types=({type_pairs})')
ctx.emit('')
def _emit_construct_method(cls: IRClass, ctx: 'EmitContext') -> None:
"""Emit the construct() body as __ct_class_NAME_construct."""
ctor = cls.constructor
bash_name = f'{CLASS_FUNC_PREFIX}{cls.name}_construct'
_emit_function_body(bash_name, ctor, ctx, this_var=True)
def _emit_method(cls: IRClass, method: IRFunction, ctx: 'EmitContext') -> None:
"""Emit a single class method."""
bash_name = f'{CLASS_FUNC_PREFIX}{cls.name}_{method.name}'
has_wrapper_dec = any(d.name in ('log', 'retry', 'cache') for d in method.decorators)
if has_wrapper_dec:
orig_name = f'{bash_name}__orig'
_emit_function_body(orig_name, method, ctx, this_var=True)
_emit_decorators_from_orig(method, bash_name, orig_name, ctx)
else:
_emit_function_body(bash_name, method, ctx, this_var=True)
_emit_decorators(method, bash_name, ctx)
def _find_array_params(fn: IRFunction) -> dict[str, str]:
"""Scan IR function body to determine which params are used as arrays/dicts."""
param_names = {p.name for p in fn.params}
result: dict[str, str] = {}
tentative: dict[str, str] = {} # provisional (ambiguous methods like get/set/len)
class_like: set[str] = set() # params with non-stdlib methods → likely class instances
_ARRAY_ONLY = {'push', 'pop', 'shift', 'slice', 'map', 'filter', 'join'}
_DICT_ONLY = {'has', 'del', 'keys'}
_AMBIGUOUS = {'len', 'get', 'set'}
_ALL_STDLIB = _ARRAY_ONLY | _DICT_ONLY | _AMBIGUOUS
visited: set[int] = set()
def visit(node) -> None:
if node is None:
return
nid = id(node)
if nid in visited:
return
visited.add(nid)
if not hasattr(node, '__dataclass_fields__'):
return
if isinstance(node, IRForeach) and isinstance(node.iterable, IRIdentifier):
name = node.iterable.name
if name in param_names:
result[name] = 'array'
if isinstance(node, IRMethodCall) and isinstance(node.receiver, IRIdentifier):
name = node.receiver.name
if name in param_names:
m = node.method_name
if m in _DICT_ONLY:
result[name] = 'dict'
elif m in _ARRAY_ONLY and result.get(name) != 'dict':
result[name] = 'array'
elif m in _AMBIGUOUS:
if name not in result:
tentative[name] = 'array'
else:
class_like.add(name)
for val in vars(node).values():
if hasattr(val, '__dataclass_fields__'):
visit(val)
elif isinstance(val, list):
for item in val:
if hasattr(item, '__dataclass_fields__'):
visit(item)
elif isinstance(item, (list, tuple)):
for sub in item:
if hasattr(sub, '__dataclass_fields__'):
visit(sub)
if fn.body:
visit(fn.body)
for name, kind in tentative.items():
if name not in result and name not in class_like:
result[name] = kind
return result
def _emit_function_body(
bash_name: str,
fn: IRFunction,
ctx: 'EmitContext',
this_var: bool = False,
) -> None:
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
param_offset = 1
if this_var:
ctx.emit('local __ct_this="$1"')
param_offset = 2
saved_array_vars = set(ctx.array_vars)
saved_dict_vars = set(ctx.dict_vars)
saved_param_array_vars = set(ctx.param_array_vars)
saved_param_dict_vars = set(ctx.param_dict_vars)
param_array_types = _find_array_params(fn)
for i, param in enumerate(fn.params):
pos = i + param_offset
if param.is_variadic:
ctx.emit(f'local -a {param.name}=("${{{pos}@}}")')
elif param.default:
default_val = _expr(param.default, ctx)
ctx.emit(f'local {param.name}="${{{pos}:-{default_val[1:-1]}}}"')
else:
ctx.emit(f'local {param.name}="${{{pos}}}"')
ptype = param_array_types.get(param.name)
if ptype == 'array':
ctx.param_array_vars.add(param.name)
elif ptype == 'dict':
ctx.param_dict_vars.add(param.name)
if fn.body:
ctx.in_function = True
emit_block(fn.body, ctx)
ctx.in_function = False
ctx.array_vars.clear()
ctx.array_vars.update(saved_array_vars)
ctx.dict_vars.clear()
ctx.dict_vars.update(saved_dict_vars)
ctx.param_array_vars.clear()
ctx.param_array_vars.update(saved_param_array_vars)
ctx.param_dict_vars.clear()
ctx.param_dict_vars.update(saved_param_dict_vars)
ctx.emit('}')
ctx.emit('')
def _emit_decorators(method: IRFunction, bash_name: str, ctx: 'EmitContext') -> None:
for dec in method.decorators:
dname = dec.name
if dec.obj_name:
# custom decorator: @router.route("/path")
args_bash = ' '.join(_expr(a, ctx) for _, a in dec.args)
ctx.emit(f'__ct_call_method "${{{dec.obj_name}}}" "{dname}" {args_bash} "{bash_name}"')
def _emit_decorators_from_orig(method: IRFunction, bash_name: str, orig: str, ctx: 'EmitContext') -> None:
for dec in method.decorators:
dname = dec.name
if dname == 'log':
_wrap_log_from_orig(bash_name, orig, ctx)
elif dname == 'retry':
attempts = 3; delay = 1
for k, v in dec.args:
if k == 'attempts': attempts = v.value if hasattr(v, 'value') else 3
elif k == 'delay': delay = v.value if hasattr(v, 'value') else 1
_wrap_retry_from_orig(bash_name, orig, attempts, delay, ctx)
elif dname == 'cache':
_wrap_cache_from_orig(bash_name, orig, ctx)
elif dec.obj_name:
args_bash = ' '.join(_expr(a, ctx) for _, a in dec.args)
ctx.emit(f'__ct_call_method "${{{dec.obj_name}}}" "{dname}" {args_bash} "{bash_name}"')
def _wrap_log(bash_name: str, ctx: 'EmitContext') -> None:
orig = f'{bash_name}__orig'
ctx.emit(f'{bash_name} () {{ echo "[LOG] {bash_name} called" >&2; {orig} "$@"; }}')
ctx.emit('')
def _wrap_log_from_orig(bash_name: str, orig: str, ctx: 'EmitContext') -> None:
ctx.emit(f'{bash_name} () {{ echo "[LOG] {bash_name} called" >&2; {orig} "$@"; }}')
ctx.emit('')
def _wrap_retry(bash_name: str, attempts: int, delay: int, ctx: 'EmitContext') -> None:
orig = f'{bash_name}__orig'
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
ctx.emit(f'local __i=0')
ctx.emit(f'while [[ $__i -lt {attempts} ]]; do')
with ctx.indented():
ctx.emit(f'{orig} "$@" && return 0')
ctx.emit(f'__i=$((__i + 1))')
ctx.emit(f'sleep {delay}')
ctx.emit(f'done')
ctx.emit(f'return 1')
ctx.emit(f'}}')
ctx.emit('')
def _wrap_retry_from_orig(bash_name: str, orig: str, attempts: int, delay: int, ctx: 'EmitContext') -> None:
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
ctx.emit(f'local __i=0')
ctx.emit(f'while [[ $__i -lt {attempts} ]]; do')
with ctx.indented():
ctx.emit(f'{orig} "$@" && return 0')
ctx.emit(f'__i=$((__i + 1))')
ctx.emit(f'sleep {delay}')
ctx.emit(f'done')
ctx.emit(f'return 1')
ctx.emit(f'}}')
ctx.emit('')
def _wrap_cache(bash_name: str, ttl: int, ctx: 'EmitContext') -> None:
orig = f'{bash_name}__orig'
ctx.emit(f'declare -gA __ct_cache_{bash_name}=()')
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
ctx.emit(f'local __key="$*"')
ctx.emit(f'if [[ -n "${{__ct_cache_{bash_name}[$__key]+x}}" ]]; then')
with ctx.indented():
ctx.emit(f'{RET_VAR}="${{__ct_cache_{bash_name}[$__key]}}"; return 0')
ctx.emit(f'fi')
ctx.emit(f'{orig} "$@"')
ctx.emit(f'__ct_cache_{bash_name}[$__key]="${{{RET_VAR}}}"')
ctx.emit('}')
ctx.emit('')
def _wrap_cache_from_orig(bash_name: str, orig: str, ctx: 'EmitContext') -> None:
ctx.emit(f'declare -gA __ct_cache_{bash_name}=()')
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
ctx.emit(f'local __key="$*"')
ctx.emit(f'if [[ -n "${{__ct_cache_{bash_name}[$__key]+x}}" ]]; then')
with ctx.indented():
ctx.emit(f'{RET_VAR}="${{__ct_cache_{bash_name}[$__key]}}"; return 0')
ctx.emit(f'fi')
ctx.emit(f'{orig} "$@"')
ctx.emit(f'__ct_cache_{bash_name}[$__key]="${{{RET_VAR}}}"')
ctx.emit('}')
ctx.emit('')
def _wrap_validate_from_orig(bash_name: str, orig: str, fn: IRFunction,
dec: IRDecorator, ctx: 'EmitContext') -> None:
import re as _re
validations: dict[str, str] = {}
for aname, aval in dec.args:
raw = _expr(aval, ctx)
validations[aname] = raw.strip('"')
ctx.emit(f'{bash_name} () {{')
with ctx.indented():
for i, param in enumerate(fn.params):
rule = validations.get(param.name, '')
if not rule:
continue
ctx.emit(f'local {param.name}="${{{i + 1}}}"')
if 'int' in rule:
ctx.emit(f'if ! [[ "${{{param.name}}}" =~ ^-?[0-9]+$ ]]; then')
with ctx.indented():
ctx.emit(f'echo "Validation error: {param.name} must be integer" >&2')
ctx.emit('return 1')
ctx.emit('fi')
for m in _re.finditer(r'(>=|<=|>|<|==|!=)\s*(-?\d+)', rule):
op, val = m.groups()
bash_op = {'>': '-gt', '<': '-lt', '>=': '-ge', '<=': '-le',
'==': '-eq', '!=': '-ne'}[op]
ctx.emit(f'if ! [[ ${{{param.name}}} {bash_op} {val} ]]; then')
with ctx.indented():
ctx.emit(f'echo "Validation error: {param.name} must be {op} {val}" >&2')
ctx.emit('return 1')
ctx.emit('fi')
elif 'string' in rule:
if 'nonempty' in rule or 'required' in rule:
ctx.emit(f'if [[ -z "${{{param.name}}}" ]]; then')
with ctx.indented():
ctx.emit(f'echo "Validation error: {param.name} cannot be empty" >&2')
ctx.emit('return 1')
ctx.emit('fi')
params_str = ' '.join(f'"${{{i + 1}}}"' for i in range(len(fn.params)))
ctx.emit(f'{orig} {params_str}')
ctx.emit('}')
ctx.emit('')
def emit_function(fn: IRFunction, ctx: 'EmitContext') -> None:
"""Emit a top-level function."""
bash_name = fn.symbol.bash_name() if fn.symbol else fn.name
if fn.is_awk:
emit_awk_function(fn, ctx)
return
has_wrapper_dec = any(d.name in ('log', 'retry', 'cache', 'validate') for d in fn.decorators)
if has_wrapper_dec:
# Emit original body as __orig, then wrap
orig_name = f'{bash_name}__orig'
_emit_function_body(orig_name, fn, ctx, this_var=False)
for dec in fn.decorators:
dname = dec.name
if dname == 'log':
_wrap_log_from_orig(bash_name, orig_name, ctx)
elif dname == 'retry':
attempts = 3; delay = 1
for aname, aval in dec.args:
if aname == 'attempts' and isinstance(aval, IRInt): attempts = aval.value
if aname == 'delay' and isinstance(aval, IRInt): delay = aval.value
_wrap_retry_from_orig(bash_name, orig_name, attempts, delay, ctx)
elif dname == 'cache':
_wrap_cache_from_orig(bash_name, orig_name, ctx)
elif dname == 'validate':
_wrap_validate_from_orig(bash_name, orig_name, fn, dec, ctx)
else:
_emit_function_body(bash_name, fn, ctx, this_var=False)
# User decorator calls are emitted as IRUserDecoratorCall in top_stmts
# (at the correct sequential position), not here.
compiler/backend/bash/constants.py 0 → 100644
View file @ 31bfe20c
from ...constants import * # noqa: F401,F403
compiler/backend/bash/expr.py 0 → 100644
View file @ 31bfe20c
"""
Expression code generation for BashBackend.
_emit_expr(node, ctx) → str (bash expression string)
_emit_expr_stmt(node, ctx) (void — emits via ctx.emit())
Context (ctx) gives access to:
ctx.emit(line) — write a bash line
ctx.in_function — bool
ctx.local_vars — set of already-declared locals
ctx.lambda_counter — for unique lambda names
ctx.emit_lambda(fn) — emit a lambda function body, return its name
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from ...ir.nodes import (
IRNode, IRInt, IRFloat, IRBool, IRNil, IRString, IRStringText, IRStringInterp,
IRArray, IRDict, IRIdentifier, IRThis, IRFieldAccess, IRIndexAccess,
IRBinaryOp, IRUnaryOp, IRCall, IRMethodCall, IRNew, IRAsync, IRLambda,
IRExpr,
)
from ...semantics.types import T_INT, T_FLOAT, T_BOOL
from .constants import (
RET_VAR, RET_ARR, OBJ_STORE, ARR_PREFIX, DICT_PREFIX, STR_PREFIX,
HTTP_PREFIX, FS_PREFIX, JSON_PREFIX, REGEX_PREFIX, MATH_PREFIX, COPROC_PREFIX,
)
if TYPE_CHECKING:
from .backend import EmitContext
# ---------------------------------------------------------------------------
# Comparison operators that need [[ ]] vs (( ))
# ---------------------------------------------------------------------------
_CMP_OPS = {'==', '!=', '<', '>', '<=', '>='}
_LOGIC_OPS = {'&&', '||'}
_ARITH_OPS = {'+', '-', '*', '/', '%', '**'}
_BASH_CMP = {'==': '==', '!=': '!=', '<': '-lt', '>': '-gt', '<=': '-le', '>=': '-ge'}
_STR_CMP = {'==': '==', '!=': '!=', '<': '<', '>': '>'}
# Stdlib method → prefix
_STDLIB_METHOD_PREFIX: dict[str, str] = {
# strings (only string methods here to avoid conflicts)
'upper': STR_PREFIX, 'lower': STR_PREFIX, 'trim': STR_PREFIX,
'len': STR_PREFIX, 'contains': STR_PREFIX, 'starts': STR_PREFIX,
'ends': STR_PREFIX, 'replace': STR_PREFIX, 'split': STR_PREFIX,
'substr': STR_PREFIX, 'index': STR_PREFIX, 'charAt': STR_PREFIX,
'urlencode': STR_PREFIX,
# arrays (non-conflicting methods only)
'push': ARR_PREFIX, 'pop': ARR_PREFIX, 'shift': ARR_PREFIX,
'join': ARR_PREFIX, 'slice': ARR_PREFIX, 'map': ARR_PREFIX,
'filter': ARR_PREFIX,
# dicts (non-conflicting methods only)
'has': DICT_PREFIX, 'del': DICT_PREFIX, 'keys': DICT_PREFIX,
# file handles
'read': '__ct_fh_', 'write': '__ct_fh_', 'writeln': '__ct_fh_',
'readline': '__ct_fh_', 'close': '__ct_fh_',
# math
'add': MATH_PREFIX, 'sub': MATH_PREFIX, 'mul': MATH_PREFIX,
'div': MATH_PREFIX, 'mod': MATH_PREFIX, 'min': MATH_PREFIX,
'max': MATH_PREFIX, 'abs': MATH_PREFIX, 'sqrt': MATH_PREFIX,
'floor': MATH_PREFIX, 'ceil': MATH_PREFIX, 'pow': MATH_PREFIX,
# regex
'match': REGEX_PREFIX, 'extract': REGEX_PREFIX,
}
# Methods whose return goes via __CT_RET_ARR (array return)
_ARR_RETURN_METHODS = {'split', 'map', 'filter', 'keys', 'slice'}
# Namespace identifiers that map to a specific category prefix
_NS_PREFIX: dict[str, str] = {
'http': HTTP_PREFIX,
'fs': FS_PREFIX,
'json': JSON_PREFIX,
'math': MATH_PREFIX,
'regex': REGEX_PREFIX,
}
# Builtin CT function names → bash function names
_BUILTIN_BASH: dict[str, str] = {
'print': '__ct_print',
'len': '__ct_len',
'range': '__ct_range',
'exit': '__ct_exit',
'pid': '__ct_pid',
'random': '__ct_random',
'random_range': '__ct_random_range',
'assert': '__ct_assert',
'assert_eq': '__ct_assert_eq',
'is_empty': '__ct_is_empty',
'is_number': '__ct_is_number',
}
# Stdlib functions that return via __CT_RET (not stdout/subshell)
_RETVAR_FUNCS = frozenset({
'upper', 'lower', 'trim', 'len', 'contains', 'starts', 'ends',
'replace', 'substr', 'index', 'charAt', 'urlencode',
'push', 'pop', 'shift', 'join', 'slice', 'get', 'set',
'has', 'del', 'keys',
'match', 'extract',
'add', 'sub', 'mul', 'div', 'mod', 'min', 'max', 'abs',
'sqrt', 'floor', 'ceil', 'pow',
'__ct_len', '__ct_range',
})
# ---------------------------------------------------------------------------
# Public entry point
# ---------------------------------------------------------------------------
def emit_expr(node: IRExpr, ctx: 'EmitContext') -> str:
"""Return bash string representing node as an rvalue."""
return _expr(node, ctx)
def emit_expr_as_stmt(node: IRExpr, ctx: 'EmitContext') -> None:
"""Emit a call/method call as a statement (no value capture)."""
if isinstance(node, IRCall):
_call_stmt(node, ctx)
elif isinstance(node, IRMethodCall):
_method_stmt(node, ctx)
elif isinstance(node, IRNew):
_new_stmt(node, ctx)
elif isinstance(node, IRBinaryOp) and node.operator == '|':
_pipe_stmt(node, ctx)
else:
bash = _expr(node, ctx)
if bash:
ctx.emit(bash)
# ---------------------------------------------------------------------------
# Core dispatcher
# ---------------------------------------------------------------------------
def _expr(node: IRExpr, ctx: 'EmitContext') -> str:
if isinstance(node, IRInt):
return str(node.value)
if isinstance(node, IRFloat):
return str(node.value)
if isinstance(node, IRBool):
return 'true' if node.value else 'false'
if isinstance(node, IRNil):
return ''
if isinstance(node, IRString):
return _string(node, ctx)
if isinstance(node, IRArray):
return _array_literal(node, ctx)
if isinstance(node, IRDict):
return _dict_literal(node, ctx)
if isinstance(node, IRIdentifier):
return _identifier(node, ctx)
if isinstance(node, IRThis):
return '"$__ct_this"'
if isinstance(node, IRFieldAccess):
return _field_access(node, ctx)
if isinstance(node, IRIndexAccess):
return _index_access(node, ctx)
if isinstance(node, IRBinaryOp):
return _binary(node, ctx)
if isinstance(node, IRUnaryOp):
return _unary(node, ctx)
if isinstance(node, IRCall):
return _call_expr(node, ctx)
if isinstance(node, IRMethodCall):
return _method_expr(node, ctx)
if isinstance(node, IRNew):
return _new_expr(node, ctx)
if isinstance(node, IRAsync):
return _async_expr(node, ctx)
if isinstance(node, IRLambda):
return _lambda_expr(node, ctx)
return '""'
# ---------------------------------------------------------------------------
# Literals
# ---------------------------------------------------------------------------
def _bash_escape(s: str) -> str:
"""Escape a string for inclusion inside double-quotes."""
return s.replace('\\', '\\\\').replace('"', '\\"').replace('`', '\\`').replace('$', '\\$')
def _string(node, ctx: 'EmitContext') -> str:
if node.is_plain:
return f'"{_bash_escape(node.plain_value)}"'
parts = []
for p in node.parts:
if isinstance(p, IRStringText):
parts.append(_bash_escape(p.text))
elif isinstance(p, IRStringInterp):
parts.append(_interp(p.expr, ctx))
return '"' + ''.join(parts) + '"'
def _interp(node: IRExpr, ctx: 'EmitContext') -> str:
"""Render an interpolated expression as a bash ${} or $(()) fragment."""
if isinstance(node, IRInt):
return str(node.value)
if isinstance(node, IRBool):
return 'true' if node.value else 'false'
if isinstance(node, IRIdentifier):
name = _var_name(node)
return f'${{{name}}}'
if isinstance(node, IRThis):
return '${__ct_this}'
if isinstance(node, IRFieldAccess):
if isinstance(node.receiver, IRIdentifier):
recv_name = _var_name(node.receiver)
cp = ctx.process_handles.get(recv_name)
if cp and node.field_name == 'pid':
return f'${cp}_pid'
recv = _interp_recv(node.receiver, ctx)
return f'${{{OBJ_STORE}["{recv}.{node.field_name}"]}}'
if isinstance(node, IRBinaryOp):
if node.operator in _ARITH_OPS:
lv = _interp_arith(node.left, ctx)
rv = _interp_arith(node.right, ctx)
return f'$(({lv} {node.operator} {rv}))'
# comparison: emit as $([[ ]] && echo true || echo false)
lb = _expr(node.left, ctx)
rb = _expr(node.right, ctx)
op = node.operator
return f'$([[ {lb} {op} {rb} ]] && echo true || echo false)'
if isinstance(node, (IRMethodCall, IRCall)):
inner = _expr(node, ctx)
if inner.startswith('$(') and inner.endswith(')'):
return inner
if inner.startswith('"') and inner.endswith('"'):
inner_val = inner[1:-1]
# If the result is __CT_RET, save to a temp to avoid it being
# overwritten by the next expression in the same interpolation.
if '__CT_RET' in inner_val:
tmp = ctx.fresh_tmp()
ctx.emit_tmp(tmp, inner)
return f'${{{tmp}}}'
return inner_val
return f'${{{inner}}}'
# fallback: generic subshell
bash = _expr(node, ctx)
if bash.startswith('"') and bash.endswith('"'):
return bash[1:-1]
return f'$({bash})'
def _interp_recv(node: IRExpr, ctx: 'EmitContext') -> str:
if isinstance(node, IRThis):
return '$__ct_this'
if isinstance(node, IRIdentifier):
return f'${{{_var_name(node)}}}'
return _expr(node, ctx).strip('"')
def _interp_arith(node: IRExpr, ctx: 'EmitContext') -> str:
if isinstance(node, IRInt):
return str(node.value)
if isinstance(node, IRIdentifier):
return _var_name(node)
if isinstance(node, IRBinaryOp) and node.operator in _ARITH_OPS:
lv = _interp_arith(node.left, ctx)
rv = _interp_arith(node.right, ctx)
return f'{lv} {node.operator} {rv}'
return _expr(node, ctx)
def _array_literal(node, ctx: 'EmitContext') -> str:
elems = ' '.join(_expr(e, ctx) for e in node.elements)
return f'({elems})'
def _dict_literal(node, ctx: 'EmitContext') -> str:
# bash assoc array init: ([key]=val ...)
pairs = ' '.join(
f'[{_expr(k, ctx)}]={_expr(v, ctx)}'
for k, v in node.pairs
)
return f'({pairs})'
# ---------------------------------------------------------------------------
# Identifiers and field access
# ---------------------------------------------------------------------------
def _var_name(node: IRIdentifier) -> str:
if node.symbol:
return node.symbol.bash_name()
return node.name
def _identifier(node: IRIdentifier, ctx: 'EmitContext') -> str:
# Function used as value (callback) → pass its bash name as a string literal
if node.symbol and node.symbol.kind == 'func':
return f'"{node.symbol.bash_name()}"'
name = _var_name(node)
# env.VAR read
if name.startswith('env.'):
env_key = name[4:]
return f'"${{{env_key}}}"'
return f'"${{{name}}}"'
def _field_access(node: IRFieldAccess, ctx: 'EmitContext') -> str:
if isinstance(node.receiver, IRThis):
return f'"${{{OBJ_STORE}["$__ct_this.{node.field_name}"]}}"'
recv = _expr(node.receiver, ctx).strip('"')
# Strip ${ } wrapper to get raw var name
if recv.startswith('${') and recv.endswith('}'):
recv = recv[2:-1]
# Process handle .pid
cp = ctx.process_handles.get(recv)
if cp and node.field_name == 'pid':
return f'"${cp}_pid"'
# Dict-as-struct: if receiver is a known dict var, use dict subscript
if recv in ctx.dict_vars or node.receiver.type.kind == 'dict':
return f'"${{{recv}["{node.field_name}"]}}"'
return f'"${{{OBJ_STORE}["${{{recv}}}.{node.field_name}"]}}"'
def _index_access(node: IRIndexAccess, ctx: 'EmitContext') -> str:
obj = _expr(node.object, ctx)
idx = _expr(node.index, ctx)
if node.object.type.kind == 'array':
# arr[idx]
obj_name = obj.strip('"').strip('${}')
idx_bare = idx.strip('"').strip('${}')
return f'"${{{obj_name}[{idx_bare}]}}"'
# dict
obj_name = obj.strip('"').strip('${}')
return f'"${{{obj_name}[{idx}]}}"'
# ---------------------------------------------------------------------------
# Operators
# ---------------------------------------------------------------------------
def _binary(node: IRBinaryOp, ctx: 'EmitContext') -> str:
op = node.operator
lt = node.left.type
rt = node.right.type
# Pipe: handle before evaluating right side (may be lazy)
if op == '|':
return _pipe(node, ctx)
lv = _expr(node.left, ctx)
# If left produced a call result in __CT_RET, spill it before right side
# (right side call would overwrite __CT_RET)
if lv == f'"${{{RET_VAR}}}"':
_tmp = ctx.fresh_tmp()
ctx.emit_tmp(_tmp, lv)
lv = f'"${{{_tmp}}}"'
rv = _expr(node.right, ctx)
# String concatenation: + with strings, or .. operator
if op == '..' or (op == '+' and (lt.kind == 'string' or rt.kind == 'string')):
lb = lv.strip('"')
rb = rv.strip('"')
return f'"{lb}{rb}"'
# Arithmetic → $(( ))
if op in _ARITH_OPS:
la = _to_arith(lv)
ra = _to_arith(rv)
if op == '/' and (lt.kind == 'float' or rt.kind == 'float'):
# float division via awk
return f'"$(echo "" | awk "BEGIN{{printf \\"%g\\", {la} / {ra}}}")"'
return f'"$(({la} {op} {ra}))"'
# Comparison
if op in _CMP_OPS:
if lt.kind in ('int', 'float') and rt.kind in ('int', 'float'):
la = _to_arith(lv)
ra = _to_arith(rv)
bash_op = _BASH_CMP[op]
return f'$([[ $(({la} {bash_op} {ra})) -ne 0 ]] && echo true || echo false)'
# string comparison
str_op = _STR_CMP.get(op, op)
return f'$([[ {lv} {str_op} {rv} ]] && echo true || echo false)'
# Logical
if op == '&&':
return f'$([[ {lv} == "true" && {rv} == "true" ]] && echo true || echo false)'
if op == '||':
return f'$([[ {lv} == "true" || {rv} == "true" ]] && echo true || echo false)'
return f'"{lv} {op} {rv}"'
def _to_arith(s: str) -> str:
"""Strip quotes/$ wrapping to get arithmetic operand."""
s = s.strip()
if s.startswith('"') and s.endswith('"'):
inner = s[1:-1]
if inner.startswith('$((') and inner.endswith('))'):
return inner[3:-2]
if inner.startswith('${') and inner.endswith('}'):
return inner
return inner
if s.startswith('$((') and s.endswith('))'):
return s[3:-2]
return s
def _unwrap_subshell(s: str) -> str:
"""Strip double-quotes and $() wrapper: '"$(cmd)"' → 'cmd'."""
s = s.strip('"')
if s.startswith('$(') and s.endswith(')'):
return s[2:-1]
return s
def _pipe_stmt(node: IRBinaryOp, ctx: 'EmitContext') -> None:
"""Emit a pipe expression in statement context (no value capture needed)."""
elements = _collect_pipe_chain(node)
has_shell = any(isinstance(e, IRCall) and e.is_shell_cmd for e in elements)
has_ct_fn = any(
(isinstance(e, IRCall) and not e.is_shell_cmd) or
(isinstance(e, IRIdentifier) and e.symbol and e.symbol.kind in ('func', 'var'))
for e in elements
)
parts = []
for elem in elements:
if isinstance(elem, IRCall) and elem.is_shell_cmd:
a = _shell_args(elem.args, ctx)
parts.append(f'{elem.callee_name} {a}' if a else elem.callee_name)
elif isinstance(elem, IRCall) and not elem.is_shell_cmd:
name = elem.callee.bash_name() if elem.callee else elem.callee_name
args = _ct_args(elem.args, ctx)
call = f'{name} {args}' if args else name
parts.append(f'{call} 3>&1' if has_shell else call)
elif isinstance(elem, IRIdentifier) and elem.symbol:
fn = elem.symbol.bash_name()
parts.append(f'{fn} 3>&1' if has_shell else fn)
else:
parts.append(_unwrap_subshell(_expr(elem, ctx)))
ctx.emit(' | '.join(parts))
def _collect_pipe_chain(node: IRBinaryOp) -> list:
"""Flatten left-associative pipe chain into a list of IRExpr elements."""
if isinstance(node.left, IRBinaryOp) and node.left.operator == '|':
return _collect_pipe_chain(node.left) + [node.right]
return [node.left, node.right]
def _pipe(node: IRBinaryOp, ctx: 'EmitContext') -> str:
"""Emit a pipe expression, handling shell-only, functional and mixed pipes."""
elements = _collect_pipe_chain(node)
# Classify each element
has_shell = any(isinstance(e, IRCall) and e.is_shell_cmd for e in elements)
has_ct_fn = any(
(isinstance(e, IRCall) and not e.is_shell_cmd) or
(isinstance(e, IRIdentifier) and e.symbol and e.symbol.kind in ('func', 'var'))
for e in elements
)
# Functional pipe: all CT functions / callbacks (no shell commands)
# Pass previous result as first argument.
if has_ct_fn and not has_shell:
result = _expr(elements[0], ctx)
for elem in elements[1:]:
if isinstance(elem, IRIdentifier) and elem.symbol:
fn = elem.symbol.bash_name()
rv = result.strip('"')
ctx.emit(f'{fn} "{rv}"')
result = f'"${{{RET_VAR}}}"'
elif isinstance(elem, IRCall):
name = elem.callee.bash_name() if elem.callee else elem.callee_name
extra = _ct_args(elem.args, ctx)
rv = result.strip('"')
sep = ' ' if extra else ''
ctx.emit(f'{name} "{rv}"{sep}{extra}')
result = f'"${{{RET_VAR}}}"'
return result
# Shell-only pipe: all shell commands (including shell.exec() etc.)
if has_shell and not has_ct_fn:
parts = []
for elem in elements:
if isinstance(elem, IRCall) and elem.is_shell_cmd:
a = _shell_args(elem.args, ctx)
parts.append(f'{elem.callee_name} {a}' if a else elem.callee_name)
else:
parts.append(_unwrap_subshell(_expr(elem, ctx)))
return f'"$({" | ".join(parts)} || true)"'
# Mixed pipe: CT functions redirect their print output via 3>&1 into the pipe.
# __ct_print uses >&3, so we need CT functions to run with 3>&1 so their
# print output reaches the pipe.
parts = []
for elem in elements:
if isinstance(elem, IRCall) and elem.is_shell_cmd:
a = _shell_args(elem.args, ctx)
parts.append(f'{elem.callee_name} {a}' if a else elem.callee_name)
elif isinstance(elem, IRCall) and not elem.is_shell_cmd:
name = elem.callee.bash_name() if elem.callee else elem.callee_name
args = _ct_args(elem.args, ctx)
call = f'{name} {args}' if args else name
parts.append(f'{call} 3>&1')
elif isinstance(elem, IRIdentifier) and elem.symbol:
fn = elem.symbol.bash_name()
parts.append(f'{fn} 3>&1')
else:
parts.append(_unwrap_subshell(_expr(elem, ctx)))
return f'"$({" | ".join(parts)} || true)"'
def _unary(node: IRUnaryOp, ctx: 'EmitContext') -> str:
operand = _expr(node.operand, ctx)
op = node.operator
if op == '!':
return f'$([[ {operand} == "true" ]] && echo false || echo true)'
if op == '-':
inner = _to_arith(operand)
return f'"$((-{inner}))"'
if op == '~':
inner = _to_arith(operand)
return f'"$((~{inner}))"'
return operand
# ---------------------------------------------------------------------------
# Calls
# ---------------------------------------------------------------------------
def _call_expr(node: IRCall, ctx: 'EmitContext') -> str:
"""Call in expression context — returns bash rvalue string."""
if node.is_shell_cmd:
mapped = _BUILTIN_BASH.get(node.callee_name)
if mapped:
args = _ct_args(node.args, ctx)
ctx.emit(f'{mapped} {args}')
return f'"${{{RET_VAR}}}"'
args = _shell_args(node.args, ctx)
cmd = f'{node.callee_name} {args}' if args else node.callee_name
return f'"$({cmd} || true)"'
name = node.callee.bash_name() if node.callee else _BUILTIN_BASH.get(node.callee_name, node.callee_name)
args = _ct_args(node.args, ctx)
# Constructor call without 'new': Logger("DEBUG") where Logger is a class
if node.callee and node.callee.kind == 'class':
tmp = ctx.fresh_tmp()
ctx.emit(f'{name} {args}' if args else name)
ctx.emit_tmp(tmp, '"$__ct_last_instance"')
return f'"${{{tmp}}}"'
# Callback via variable or function parameter holding a lambda/func name
if node.callee and node.callee.kind in ('var', 'param'):
ctx.emit(f'"${{{name}}}" {args}' if args else f'"${{{name}}}"')
return f'"${{{RET_VAR}}}"'
# Callback via expression: (lambda)(args) → __callback__ built in IR builder
if name == '__callback__':
cb = _expr(node.args[0], ctx)
cb_fn = cb.strip('"')
rest = _ct_args(node.args[1:], ctx)
if cb_fn.startswith('${') and cb_fn.endswith('}'):
ctx.emit(f'"{cb_fn}" {rest}' if rest else f'"{cb_fn}"')
else:
ctx.emit(f'{cb_fn} {rest}' if rest else cb_fn)
return f'"${{{RET_VAR}}}"'
# pid() → inline "$$"
if name == '__ct_pid':
return '"$$"'
# Builtins that set __CT_RET
if name in ('__ct_len', '__ct_random', '__ct_random_range'):
ctx.emit(f'{name} {args}')
return f'"${{{RET_VAR}}}"'
# range() — returns sequence via stdout
if name == '__ct_range':
return f'$({name} {args})'
# print — no useful return
if name == '__ct_print':
ctx.emit(f'{name} {args}')
return '""'
# assert
if name in ('__ct_assert', '__ct_assert_eq'):
ctx.emit(f'{name} {args}')
return '""'
# Generic CT function call: sets __CT_RET
ctx.emit(f'{name} {args}')
return f'"${{{RET_VAR}}}"'
def _call_stmt(node: IRCall, ctx: 'EmitContext') -> None:
"""Call in statement context — no value capture."""
if node.is_shell_cmd:
mapped = _BUILTIN_BASH.get(node.callee_name)
if mapped:
args = _ct_args(node.args, ctx)
ctx.emit(f'{mapped} {args}')
return
args = _shell_args(node.args, ctx)
ctx.emit(f'{node.callee_name} {args}' if args else node.callee_name)
return
name = node.callee.bash_name() if node.callee else _BUILTIN_BASH.get(node.callee_name, node.callee_name)
args = _ct_args(node.args, ctx)
# Constructor call without 'new' — discard return value
if node.callee and node.callee.kind == 'class':
ctx.emit(f'{name} {args}' if args else name)
return
# Callback via variable or function parameter
if node.callee and node.callee.kind in ('var', 'param'):
ctx.emit(f'"${{{name}}}" {args}' if args else f'"${{{name}}}"')
return
# Callback via expression
if name == '__callback__':
cb = _expr(node.args[0], ctx)
cb_fn = cb.strip('"')
rest = _ct_args(node.args[1:], ctx)
if cb_fn.startswith('${') and cb_fn.endswith('}'):
ctx.emit(f'"{cb_fn}" {rest}' if rest else f'"{cb_fn}"')
else:
ctx.emit(f'{cb_fn} {rest}' if rest else cb_fn)
return
ctx.emit(f'{name} {args}' if args else name)
def _method_expr(node: IRMethodCall, ctx: 'EmitContext') -> str:
"""Method call in expression context."""
method = node.method_name
if node.kind == 'stdlib':
return _stdlib_method_expr(node, ctx)
if node.kind == 'instance':
return _instance_method_expr(node, ctx)
# shell method (namespace call like http.get)
recv = _expr(node.receiver, ctx)
ns = recv.strip('"').strip('${}')
prefix = _NS_PREFIX.get(ns, '__ct_' + ns + '_')
args = _ct_args(node.args, ctx)
fn = f'{prefix}{method}'
ctx.emit(f'{fn} {args}')
return f'"${{{RET_VAR}}}"'
def _method_stmt(node: IRMethodCall, ctx: 'EmitContext') -> None:
"""Method call in statement context."""
method = node.method_name
if node.kind == 'stdlib':
_stdlib_method_stmt(node, ctx)
return
if node.kind == 'instance':
recv = _expr(node.receiver, ctx)
recv_name = recv.strip('"').strip('${}')
# Process handle methods (kill, wait, etc.)
cp = ctx.process_handles.get(recv_name)
if cp and _process_handle_stmt(cp, method, node.args, ctx):
return
args = _ct_args(node.args, ctx)
_DICT_ONLY_M_ = {'has', 'del', 'keys'}
is_field_recv_ = isinstance(node.receiver, IRFieldAccess)
is_param_array = recv_name in ctx.param_array_vars
is_actual_array = recv_name in ctx.array_vars
is_field_array = is_field_recv_ and method not in _DICT_ONLY_M_
if is_param_array or is_actual_array or is_field_array:
arr_ref = recv if (is_param_array or is_field_array) else f'"{recv_name}"'
ctx.emit(f'{ARR_PREFIX}{method} {arr_ref} {args}'.strip())
return
is_param_dict = recv_name in ctx.param_dict_vars
is_actual_dict = recv_name in ctx.dict_vars
is_field_dict = is_field_recv_ and method in _DICT_ONLY_M_
if is_param_dict or is_actual_dict or is_field_dict:
dict_ref = recv if (is_param_dict or is_field_dict) else f'"{recv_name}"'
ctx.emit(f'{DICT_PREFIX}{method} {dict_ref} {args}'.strip())
return
cls_hint = ''
if node.receiver.type.kind == 'class':
cls_hint = node.receiver.type.class_name or ''
if cls_hint:
ctx.emit(f'__ct_class_{cls_hint}_{method} "${{{recv_name}}}" {args}')
else:
ctx.emit(f'__ct_call_method "${{{recv_name}}}" "{method}" {args}')
return
# namespace/shell
recv = _expr(node.receiver, ctx)
ns = recv.strip('"').strip('${}')
prefix = _NS_PREFIX.get(ns, '__ct_' + ns + '_')
args = _ct_args(node.args, ctx)
ctx.emit(f'{prefix}{method} {args}')
_NS_METHOD_PREFIX: dict[str, str] = {
'http': HTTP_PREFIX,
'fs': FS_PREFIX,
'json': JSON_PREFIX,
'math': MATH_PREFIX,
'regex': REGEX_PREFIX,
'logger': '__ct_logger_',
'reflect': '__ct_reflect_',
'time': '__ct_time_',
'args': '__ct_args_',
'shell': '__ct_shell_',
}
def _stdlib_method_expr(node: IRMethodCall, ctx: 'EmitContext') -> str:
method = node.method_name
recv = _expr(node.receiver, ctx)
# Process handle methods (write, close, kill, read, wait)
recv_bare = recv.strip('"').strip('${}')
cp = ctx.process_handles.get(recv_bare)
if cp:
result = _process_handle_expr(cp, method, node.args, ctx)
if result is not None:
return result
# Namespace methods: logger.info(), fs.read(), http.get(), etc.
if isinstance(node.receiver, IRIdentifier):
ns = node.receiver.name
# shell.exec("cmd") / shell.capture("cmd") → $(eval "$1")
if ns == 'shell' and method in ('exec', 'capture', 'source'):
if node.args:
cmd = _expr(node.args[0], ctx).strip('"')
return f'"$({cmd})"'
return '""'
if ns in _NS_METHOD_PREFIX:
ns_prefix = _NS_METHOD_PREFIX[ns]
fn = f'{ns_prefix}{method}'
args = _ct_args(node.args, ctx)
ctx.emit(f'{fn} {args}'.strip())
return f'"${{{RET_VAR}}}"'
recv_name = recv.strip('"').strip('${}')
is_field_recv = isinstance(node.receiver, IRFieldAccess)
is_known_array = node.receiver.type.kind == 'array'
is_actual_array = recv_name in ctx.array_vars # actual array var — pass by name
is_param_array = recv_name in ctx.param_array_vars # param holding array name — pass "${p}"
# Field access holding array name — unambiguous array methods, or ambiguous (heuristic: default array)
_DICT_ONLY_M = {'has', 'del', 'keys'}
is_field_array = is_field_recv and method not in _DICT_ONLY_M
is_known_dict = node.receiver.type.kind == 'dict'
is_actual_dict = recv_name in ctx.dict_vars
is_param_dict = recv_name in ctx.param_dict_vars
is_field_dict = is_field_recv and method in _DICT_ONLY_M
# arr.len() fast path — only for statically-typed or actual local arrays (not field access)
if method == 'len' and (is_known_array or is_actual_array) and not is_field_recv:
return '"${#' + recv_name + '[@]}"'
args = _ct_args(node.args, ctx)
# Array path — field access uses recv directly (it expands to the array name)
if is_known_array or is_actual_array or is_param_array or is_field_array:
if is_param_array or is_field_array:
arr_ref = recv # "${param}" or "${__CT_OBJ[...]}" expands to array name
else:
arr_ref = f'"{recv_name}"' # literal array variable name
fn = f'{ARR_PREFIX}{method}'
call_args = f'{arr_ref} {args}'.strip()
if method in _ARR_RETURN_METHODS:
ctx.emit(f'{fn} {call_args}')
return f'"${{{RET_ARR}[@]}}"'
ctx.emit(f'{fn} {call_args}')
return f'"${{{RET_VAR}}}"'
# Dict path
if is_known_dict or is_actual_dict or is_param_dict or is_field_dict:
if is_param_dict or is_field_dict:
dict_ref = recv
else:
dict_ref = f'"{recv_name}"'
dict_prefix = DICT_PREFIX if method in ('get', 'set', 'has', 'del', 'keys') else ARR_PREFIX
fn = f'{dict_prefix}{method}'
call_args = f'{dict_ref} {args}'.strip()
if method in _ARR_RETURN_METHODS:
ctx.emit(f'{fn} {call_args}')
return f'"${{{RET_ARR}[@]}}"'
ctx.emit(f'{fn} {call_args}')
return f'"${{{RET_VAR}}}"'
# String methods (and fallback)
prefix = _STDLIB_METHOD_PREFIX.get(method, STR_PREFIX)
fn = f'{prefix}{method}'
call_args = f'{recv} {args}'.strip()
ctx.emit(f'{fn} {call_args}')
if method in _ARR_RETURN_METHODS:
return f'"${{{RET_ARR}[@]}}"'
return f'"${{{RET_VAR}}}"'
def _stdlib_method_stmt(node: IRMethodCall, ctx: 'EmitContext') -> None:
method = node.method_name
recv = _expr(node.receiver, ctx)
# Process handle methods (write, close, kill, read, wait)
recv_bare = recv.strip('"').strip('${}')
cp = ctx.process_handles.get(recv_bare)
if cp and _process_handle_stmt(cp, method, node.args, ctx):
return
# Namespace methods: logger.info(), fs.write(), etc.
if isinstance(node.receiver, IRIdentifier):
ns = node.receiver.name
if ns in _NS_METHOD_PREFIX:
ns_prefix = _NS_METHOD_PREFIX[ns]
fn = f'{ns_prefix}{method}'
args = _ct_args(node.args, ctx)
ctx.emit(f'{fn} {args}'.strip())
return
recv_name = recv.strip('"').strip('${}')
is_field_recv = isinstance(node.receiver, IRFieldAccess)
is_known_array = node.receiver.type.kind == 'array'
is_actual_array = recv_name in ctx.array_vars
is_param_array = recv_name in ctx.param_array_vars
_DICT_ONLY_M = {'has', 'del', 'keys'}
is_field_array = is_field_recv and method not in _DICT_ONLY_M
is_known_dict = node.receiver.type.kind == 'dict'
is_actual_dict = recv_name in ctx.dict_vars
is_param_dict = recv_name in ctx.param_dict_vars
is_field_dict = is_field_recv and method in _DICT_ONLY_M
# Fast-path mutations only for statically-typed or local arrays (not params or field access)
if (is_known_array or is_actual_array) and not is_field_recv:
if method == 'push':
args_list = [_expr(a, ctx) for a in node.args]
ctx.emit(f'{recv_name}+=({" ".join(args_list)})')
return
if method == 'pop':
ctx.emit(f'unset "{recv_name}[-1]"')
return
if method == 'shift':
ctx.emit(f'{recv_name}=("${{{recv_name}[@]:1}}")')
return
if (is_known_dict or is_actual_dict) and not is_field_recv:
if method == 'set':
k = _expr(node.args[0], ctx)
v = _expr(node.args[1], ctx)
ctx.emit(f'{recv_name}[{k}]={v}')
return
if method == 'del':
k = _expr(node.args[0], ctx)
ctx.emit(f'unset {recv_name}[{k}]')
return
# Generic dispatch
args = _ct_args(node.args, ctx)
if is_known_array or is_actual_array or is_param_array or is_field_array:
arr_ref = recv if (is_param_array or is_field_array) else f'"{recv_name}"'
fn = f'{ARR_PREFIX}{method}'
ctx.emit(f'{fn} {arr_ref} {args}'.strip())
elif is_known_dict or is_actual_dict or is_param_dict or is_field_dict:
dict_ref = recv if (is_param_dict or is_field_dict) else f'"{recv_name}"'
fn = f'{DICT_PREFIX}{method}'
ctx.emit(f'{fn} {dict_ref} {args}'.strip())
else:
prefix = _STDLIB_METHOD_PREFIX.get(method, STR_PREFIX)
fn = f'{prefix}{method}'
ctx.emit(f'{fn} {recv} {args}'.strip())
def _instance_method_expr(node: IRMethodCall, ctx: 'EmitContext') -> str:
recv = _expr(node.receiver, ctx)
recv_name = recv.strip('"').strip('${}')
method = node.method_name
args = _ct_args(node.args, ctx)
# Process handle methods (kill, wait, etc.)
cp = ctx.process_handles.get(recv_name)
if cp:
result = _process_handle_expr(cp, method, node.args, ctx)
if result is not None:
return result
_DICT_ONLY_M = {'has', 'del', 'keys'}
is_field_recv = isinstance(node.receiver, IRFieldAccess)
is_param_array = recv_name in ctx.param_array_vars
is_actual_array = recv_name in ctx.array_vars
is_field_array = is_field_recv and method not in _DICT_ONLY_M
if is_param_array or is_actual_array or is_field_array:
arr_ref = recv if (is_param_array or is_field_array) else f'"{recv_name}"'
fn = f'{ARR_PREFIX}{method}'
call_args = f'{arr_ref} {args}'.strip()
if method in _ARR_RETURN_METHODS:
ctx.emit(f'{fn} {call_args}')
return f'"${{{RET_ARR}[@]}}"'
ctx.emit(f'{fn} {call_args}')
return f'"${{{RET_VAR}}}"'
is_param_dict = recv_name in ctx.param_dict_vars
is_actual_dict = recv_name in ctx.dict_vars
is_field_dict = is_field_recv and method in _DICT_ONLY_M
if is_param_dict or is_actual_dict or is_field_dict:
dict_ref = recv if (is_param_dict or is_field_dict) else f'"{recv_name}"'
fn = f'{DICT_PREFIX}{method}'
ctx.emit(f'{fn} {dict_ref} {args}'.strip())
return f'"${{{RET_VAR}}}"'
cls_hint = ''
if node.receiver.type.kind == 'class':
cls_hint = node.receiver.type.class_name or ''
if cls_hint:
ctx.emit(f'__ct_class_{cls_hint}_{method} "${{{recv_name}}}" {args}')
else:
ctx.emit(f'__ct_call_method "${{{recv_name}}}" "{method}" {args}')
return f'"${{{RET_VAR}}}"'
def _process_handle_stmt(cp: str, method: str, args, ctx: 'EmitContext') -> bool:
"""Emit a process-handle method call in statement context. Returns True if handled."""
if method == 'write':
for a in args:
val = _expr(a, ctx)
ctx.emit(f'echo {val} >&${cp}_wr')
return True
if method == 'close':
ctx.emit(f'exec {{{cp}_wr}}>&-')
return True
if method == 'kill':
ctx.emit(f'exec {{{cp}_wr}}>&- 2>/dev/null || :')
ctx.emit(f'exec {{{cp}_rd}}<&- 2>/dev/null || :')
ctx.emit(f'kill ${cp}_pid 2>/dev/null || true')
return True
if method == 'wait':
ctx.emit(f'exec {{{cp}_wr}}>&- 2>/dev/null || :')
ctx.emit(f'exec {{{cp}_rd}}<&- 2>/dev/null || :')
ctx.emit(f'wait ${cp}_pid 2>/dev/null || true')
return True
if method == 'read':
ctx.emit(f'IFS= read -r {RET_VAR} <&${cp}_rd')
return True
return False
def _process_handle_expr(cp: str, method: str, args, ctx: 'EmitContext') -> str | None:
"""Emit a process-handle method call in expression context. Returns str or None."""
if method == 'write':
for a in args:
val = _expr(a, ctx)
ctx.emit(f'echo {val} >&${cp}_wr')
return '""'
if method == 'close':
ctx.emit(f'exec {{{cp}_wr}}>&-')
return '""'
if method == 'kill':
ctx.emit(f'exec {{{cp}_wr}}>&- 2>/dev/null || :')
ctx.emit(f'exec {{{cp}_rd}}<&- 2>/dev/null || :')
ctx.emit(f'kill ${cp}_pid 2>/dev/null || true')
return '""'
if method == 'wait':
ctx.emit(f'exec {{{cp}_wr}}>&- 2>/dev/null || :')
ctx.emit(f'exec {{{cp}_rd}}<&- 2>/dev/null || :')
ctx.emit(f'wait ${cp}_pid 2>/dev/null || true')
return '""'
if method == 'read':
ctx.emit(f'IFS= read -r {RET_VAR} <&${cp}_rd')
return f'"${{{RET_VAR}}}"'
return None
def _new_expr(node: IRNew, ctx: 'EmitContext') -> str:
cls = node.class_sym.bash_name() if node.class_sym else node.class_name
args = _ct_args(node.args, ctx)
tmp = ctx.fresh_tmp()
ctx.emit(f'{cls} {args}')
ctx.emit_tmp(tmp, '"$__ct_last_instance"')
return f'"${{{tmp}}}"'
def _new_stmt(node: IRNew, ctx: 'EmitContext') -> None:
cls = node.class_sym.bash_name() if node.class_sym else node.class_name
args = _ct_args(node.args, ctx)
ctx.emit(f'{cls} {args}')
def _async_expr(node: IRAsync, ctx: 'EmitContext') -> str:
inner = node.expr
if isinstance(inner, IRCall):
if inner.is_shell_cmd:
cmd = inner.callee_name
args = _shell_args(inner.args, ctx)
name = ctx.fresh_coproc()
ctx.emit(f'coproc {name} ({cmd} {args})')
else:
cmd = inner.callee.bash_name() if inner.callee else inner.callee_name
args = _ct_args(inner.args, ctx)
name = ctx.fresh_coproc()
ctx.emit(f'coproc {name} ({cmd} {args})')
tmp = ctx.fresh_tmp()
ctx.emit(f'declare -gA {tmp}=([in]="${{{name}[1]}}" [out]="${{{name}[0]}}" [pid]="${{{name}_PID}}")')
return f'"${{{tmp}}}"'
# fallback
bash = _expr(inner, ctx)
return f'"$({bash})"'
def _lambda_expr(node: IRLambda, ctx: 'EmitContext') -> str:
"""Emit lambda as a named bash function and return its name."""
name = ctx.emit_lambda(node)
return f'"{name}"'
# ---------------------------------------------------------------------------
# Argument helpers
# ---------------------------------------------------------------------------
def _ct_args(args, ctx: 'EmitContext') -> str:
"""Render CT function arguments. Array/dict variables are passed by name."""
parts = []
for a in args:
if isinstance(a, IRDict):
tmp = ctx.fresh_tmp()
pairs = ' '.join(f'[{_expr(k, ctx)}]={_expr(v, ctx)}' for k, v in a.pairs)
ctx.emit(f'declare -A {tmp}=({pairs})')
parts.append(f'"{tmp}"')
elif isinstance(a, IRArray):
tmp = ctx.fresh_tmp()
elems = ' '.join(_expr(e, ctx) for e in a.elements)
ctx.emit(f'local -a {tmp}=({elems})')
parts.append(f'"{tmp}"')
elif isinstance(a, IRIdentifier):
vname = _var_name(a)
if a.type.kind in ('array', 'dict'):
# Statically-typed array/dict — pass by name
parts.append(f'"{vname}"')
elif vname in ctx.array_vars or vname in ctx.dict_vars:
# Actual local array/dict var — pass by name
parts.append(f'"{vname}"')
elif vname in ctx.param_array_vars or vname in ctx.param_dict_vars:
# Param holding array/dict name — pass "${param}" (expands to name at runtime)
parts.append(_expr(a, ctx))
else:
parts.append(_expr(a, ctx))
else:
parts.append(_expr(a, ctx))
return ' '.join(parts)
def _shell_args(args, ctx: 'EmitContext') -> str:
"""Render shell command arguments."""
parts = []
for a in args:
v = _expr(a, ctx)
parts.append(v)
return ' '.join(parts)
compiler/backend/bash/stdlib.py 0 → 100644
View file @ 31bfe20c
"""
Standard library emission for BashBackend.
Re-uses bootstrap/methods/* as the single source of truth for bash implementations.
Only emits categories that DCE determined are actually used.
"""
from __future__ import annotations
from ...methods import (
STRING_METHODS, ARRAY_METHODS, DICT_METHODS, FILE_HANDLE_METHODS,
HTTP_METHODS, FS_METHODS, JSON_METHODS, LOGGER_METHODS,
REGEX_METHODS, MATH_METHODS, TIME_METHODS, ARGS_METHODS, CORE_FUNCTIONS,
)
def emit_stdlib(out: list[str], used_categories: set[str], indent: str = '') -> None:
"""Emit stdlib bash functions into out list."""
em = _Emitter(out, indent)
em.line('# === ContenT Standard Library ===')
em.blank()
_emit_core(em)
cats = used_categories
if 'object' in cats or _has_classes(cats):
_emit_object_system(em)
if 'exception' in cats:
_emit_exception(em)
if 'http' in cats:
_emit_http(em)
if 'fs' in cats:
_emit_fs(em)
if 'json' in cats:
_emit_json(em)
if 'logger' in cats:
_emit_logger(em)
if 'string' in cats:
_emit_string(em)
if 'array' in cats:
_emit_array(em)
if 'dict' in cats:
_emit_dict(em)
if 'regex' in cats:
_emit_regex(em)
if 'reflect' in cats:
_emit_reflect(em)
if 'awk' in cats or 'math' in cats:
_emit_awk_wrapper(em)
if 'math' in cats:
_emit_math(em)
if 'misc' in cats or 'time' in cats:
_emit_misc(em)
if 'args' in cats:
_emit_args(em)
else:
_emit_utils_minimal(em)
if 'test' in cats:
_emit_test(em)
if 'misc' in cats:
_emit_busing_misc(em)
em.line('# === End Standard Library ===')
em.blank()
def _has_classes(cats: set[str]) -> bool:
return bool(cats - {'core', 'awk', 'math', 'time', 'args', 'misc', 'test'})
# ---------------------------------------------------------------------------
# Thin emitter helper
# ---------------------------------------------------------------------------
class _Emitter:
def __init__(self, out: list[str], base_indent: str = '') -> None:
self._out = out
self._base = base_indent
self._lvl = 0
def line(self, s: str = '') -> None:
if s:
self._out.append(self._base + ' ' * self._lvl + s)
else:
self._out.append('')
def blank(self) -> None:
self._out.append('')
def indent(self) -> None:
self._lvl += 1
def dedent(self) -> None:
self._lvl -= 1
# ---------------------------------------------------------------------------
# Section emitters
# ---------------------------------------------------------------------------
def _emit_core(em: _Emitter) -> None:
em.line("exec 3>&1")
em.blank()
em.line(f"{CORE_FUNCTIONS['print'].bash_func} () {{ {CORE_FUNCTIONS['print'].bash_impl}; }}")
em.blank()
em.line("__ct_range () {")
em.indent()
em.line('local start=0 end step=1')
em.line('case $# in')
em.indent()
em.line('1) end=$1 ;;')
em.line('2) start=$1; end=$2 ;;')
em.line('3) start=$1; end=$2; step=$3 ;;')
em.dedent()
em.line('esac')
em.line('seq "$start" "$step" "$((end - 1))"')
em.dedent()
em.line("}")
em.blank()
em.line("__ct_len () {")
em.indent()
em.line('local __d; __d=$(declare -p "$1" 2>/dev/null) && [[ "$__d" == "declare -a"* ]] && {')
em.indent()
em.line('local -n __a="$1"; __CT_RET=${#__a[@]}; echo "$__CT_RET"; return')
em.dedent()
em.line('}')
em.line('__CT_RET=${#1}; echo "$__CT_RET"')
em.dedent()
em.line("}")
em.blank()
em.line(f"{CORE_FUNCTIONS['pid'].bash_func} () {{ {CORE_FUNCTIONS['pid'].bash_impl}; }}")
em.blank()
em.line("declare -g __CT_RET=''")
em.line("declare -ga __CT_RET_ARR=()")
em.blank()
def _emit_object_system(em: _Emitter) -> None:
em.line("declare -gA __ct_obj_class=()")
em.line("declare -gA __CT_OBJ=()")
em.line("declare -g __ct_last_instance=''")
em.blank()
em.line("__ct_call_method () {")
em.indent()
em.line('local __obj="$1" __method="$2"')
em.line('shift 2')
em.line('local __class="${__ct_obj_class[$__obj]:-}"')
em.line('[[ -z "$__class" ]] && { echo "Error: Unknown object $__obj" >&2; return 1; }')
em.line('"__ct_class_${__class}_${__method}" "$__obj" "$@"')
em.dedent()
em.line("}")
em.blank()
em.line('__ct_get_field () { echo "${__CT_OBJ[$1.$2]}"; }')
em.blank()
def _emit_exception(em: _Emitter) -> None:
em.line("declare -g __ct_exception=''")
em.line("declare -g __ct_exception_type=''")
em.blank()
em.line("__ct_throw () {")
em.indent()
em.line('declare -g __ct_exception_type="$1"')
em.line('declare -g __ct_exception="$2"')
em.line('return 1')
em.dedent()
em.line("}")
em.blank()
def _emit_http(em: _Emitter) -> None:
for m in HTTP_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_to_ret(m.bash_impl)}; }}")
em.blank()
def _emit_fs(em: _Emitter) -> None:
for m in FS_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_to_ret(m.bash_impl)}; }}")
em.blank()
# File handle system
em.line("declare -gA __ct_file_handles=()")
em.line("declare -g __ct_file_fd=10")
em.blank()
em.line("__ct_fs_open () {")
em.indent()
em.line('local path="$1" mode="${2:-r}" fd=$__ct_file_fd')
em.line('__ct_file_fd=$((fd + 1))')
em.line('local h="__fh_${fd}"')
em.line('__ct_file_handles["${h}_path"]="$path"')
em.line('__ct_file_handles["${h}_fd"]="$fd"')
em.line('__ct_file_handles["${h}_mode"]="$mode"')
em.line('__CT_RET="$h"')
em.dedent()
em.line("}")
em.blank()
for m in FILE_HANDLE_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_no_echo(m.bash_impl)}; }}")
em.blank()
em.line('__ct_fh___enter__ () { echo "$1"; }')
em.line('__ct_fh___exit__ () { __ct_fh_close "$1"; }')
em.blank()
def _emit_json(em: _Emitter) -> None:
em.line("__ct_json_parse () {")
em.indent()
em.line('local __json="$1" __dict_name="$2"')
em.line('eval "declare -gA $__dict_name=()"')
em.line('while IFS= read -r __line; do')
em.indent()
em.line('local __key="${__line%%=*}" __val="${__line#*=}"')
em.line('[[ -n "$__key" ]] && eval "${__dict_name}[\\"$__key\\"]=\"\\$__val\""')
em.dedent()
em.line('done < <(echo "$__json" | jq -r \'to_entries[] | "\\(.key)=\\(.value)"\' 2>/dev/null)')
em.dedent()
em.line("}")
em.blank()
em.line("__ct_json_get () { __CT_RET=$(echo \"$1\" | jq -r \"$2\" 2>/dev/null); }")
em.blank()
em.line("__ct_json_unmarshal () {")
em.indent()
em.line('local __json="$1" __class="$2"')
em.line('"$__class"')
em.line('local __obj="$__ct_last_instance"')
em.line('local __cls="${__ct_obj_class[$__obj]}"')
em.line('local -n __fields="__ct_class_meta_${__cls}_fields"')
em.line('local -n __types="__ct_class_meta_${__cls}_types"')
em.line('local __f')
em.line('for __f in "${__fields[@]}"; do')
em.indent()
em.line('local __t="${__types[$__f]}" __val')
em.line('if declare -p "__ct_class_meta_${__t}_fields" &>/dev/null; then')
em.indent()
em.line('__val="$(echo "$__json" | jq -c --arg f "$__f" \'.[$f] // empty\' 2>/dev/null)"')
em.line('[[ -n "$__val" && "$__val" != "null" ]] && { __ct_json_unmarshal "$__val" "$__t"; __CT_OBJ["$__obj.$__f"]="$__ct_last_instance"; }')
em.dedent()
em.line('else')
em.indent()
em.line('__val="$(echo "$__json" | jq -r --arg f "$__f" \'.[$f] // empty\' 2>/dev/null)"')
em.line('[[ -n "$__val" ]] && __CT_OBJ["$__obj.$__f"]="$__val"')
em.dedent()
em.line('fi')
em.dedent()
em.line('done')
em.line('__ct_last_instance="$__obj"')
em.dedent()
em.line("}")
em.blank()
em.line("__ct_json_marshal () {")
em.indent()
em.line('local __obj="$1" __cls __first=1')
em.line('__cls="${__ct_obj_class[$__obj]}"')
em.line('local -n __fields="__ct_class_meta_${__cls}_fields"')
em.line('local -n __types="__ct_class_meta_${__cls}_types"')
em.line('printf "{"')
em.line('for __f in "${__fields[@]}"; do')
em.indent()
em.line('[[ $__first -eq 1 ]] && __first=0 || printf ","')
em.line('printf "\\"%s\\":" "$__f"')
em.line('local __v="${__CT_OBJ["$__obj.$__f"]}" __t="${__types[$__f]}"')
em.line('if [[ -n "${__ct_obj_class[$__v]+x}" ]]; then __ct_json_marshal "$__v"')
em.line('elif [[ "$__t" == "int" || "$__t" == "float" || "$__t" == "bool" ]]; then printf "%s" "$__v"')
em.line('else printf "\\"%s\\"" "$__v"; fi')
em.dedent()
em.line('done')
em.line('printf "}"')
em.dedent()
em.line("}")
em.blank()
em.line("__ct_json_stringify () {")
em.indent()
em.line('local -n __d="$1"; local __first=1')
em.line('printf "{"')
em.line('for __k in "${!__d[@]}"; do')
em.indent()
em.line('[[ $__first -eq 1 ]] && __first=0 || printf ","')
em.line('printf "\\"%s\\":" "$__k"')
em.line('local __v="${__d[$__k]}"')
em.line('if [[ "$__v" =~ ^-?[0-9]+$ ]] || [[ "$__v" =~ ^(true|false|null)$ ]]; then printf "%s" "$__v"')
em.line('else printf "\\"%s\\"" "$__v"; fi')
em.dedent()
em.line('done')
em.line('printf "}\\n"')
em.dedent()
em.line("}")
em.blank()
def _emit_logger(em: _Emitter) -> None:
for m in LOGGER_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {m.bash_impl}; }}")
em.blank()
def _no_echo(impl: str) -> str:
"""Strip trailing echo of __CT_RET from bash_impl (new backend uses inline __CT_RET pattern)."""
import re
return re.sub(r';\s*echo\s+"?\$\{?__CT_RET\}?"?\s*$', '', impl).rstrip('; ')
def _to_ret(impl: str) -> str:
"""Make bash_impl set __CT_RET. If it already sets __CT_RET, strip echo. Otherwise wrap."""
if '__CT_RET' in impl:
return _no_echo(impl)
return f'__CT_RET=$({impl})'
def _emit_string(em: _Emitter) -> None:
for m in STRING_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_no_echo(m.bash_impl)}; }}")
em.blank()
def _emit_array(em: _Emitter) -> None:
for m in ARRAY_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_no_echo(m.bash_impl)}; }}")
em.blank()
em.line("__ct_arr_map () {")
em.indent()
em.line('local -n __a=$1; local __fn=$2; local -a __result=()')
em.line('for __item in "${__a[@]}"; do "$__fn" "$__item"; __result+=("$__CT_RET"); done')
em.line('__CT_RET_ARR=("${__result[@]}")')
em.dedent()
em.line("}")
em.blank()
em.line("__ct_arr_filter () {")
em.indent()
em.line('local -n __a=$1; local __fn=$2; local -a __result=()')
em.line('for __item in "${__a[@]}"; do')
em.indent()
em.line('"$__fn" "$__item"; [[ "$__CT_RET" == "true" ]] && __result+=("$__item")')
em.dedent()
em.line('done')
em.line('__CT_RET_ARR=("${__result[@]}")')
em.dedent()
em.line("}")
em.blank()
def _emit_dict(em: _Emitter) -> None:
for m in DICT_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_no_echo(m.bash_impl)}; }}")
em.blank()
def _emit_regex(em: _Emitter) -> None:
for m in REGEX_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_to_ret(m.bash_impl)}; }}")
em.blank()
def _emit_reflect(em: _Emitter) -> None:
em.line("__ct_reflect_fields () {")
em.indent()
em.line('local __obj="$1" __cls="${__ct_obj_class[$1]}"')
em.line('local -n __fields="__ct_class_meta_${__cls}_fields"')
em.line('__CT_RET_ARR=("${__fields[@]}")')
em.dedent()
em.line("}")
em.blank()
em.line("__ct_reflect_type () {")
em.indent()
em.line('local __obj="$1" __field="$2" __cls="${__ct_obj_class[$1]}"')
em.line('local -n __types="__ct_class_meta_${__cls}_types"')
em.line('__CT_RET="${__types[$__field]}"')
em.dedent()
em.line("}")
em.blank()
em.line('__ct_reflect_get () { local __obj="$1" __field="$2"; __CT_RET="${__CT_OBJ["$__obj.$__field"]}"; }')
em.line('__ct_reflect_set () { __CT_OBJ["$1.$2"]="$3"; }')
em.line('__ct_reflect_class_name () { __CT_RET="${__ct_obj_class[$1]}"; }')
em.line('__ct_reflect_create () { local __class="$1"; shift; "$__class" "$@"; }')
em.blank()
def _emit_awk_wrapper(em: _Emitter) -> None:
em.line("__ct_awk_cmd=''")
em.line("if command -v mawk &>/dev/null; then __ct_awk_cmd=mawk")
em.line("elif command -v gawk &>/dev/null; then __ct_awk_cmd=gawk")
em.line("elif command -v awk &>/dev/null; then __ct_awk_cmd=awk")
em.line("else echo '[ERROR] No awk found' >&2; exit 1; fi")
em.line('__ct_awk () { "$__ct_awk_cmd" "$@"; }')
em.blank()
def _emit_math(em: _Emitter) -> None:
for m in MATH_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_to_ret(m.bash_impl)}; }}")
em.blank()
def _emit_misc(em: _Emitter) -> None:
em.line("__CT_NL=$'\\n'")
for m in TIME_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_to_ret(m.bash_impl)}; }}")
em.line(f"{CORE_FUNCTIONS['random'].bash_func} () {{ {_to_ret(CORE_FUNCTIONS['random'].bash_impl)}; }}")
em.line(f"{CORE_FUNCTIONS['random_range'].bash_func} () {{ {_to_ret(CORE_FUNCTIONS['random_range'].bash_impl)}; }}")
em.blank()
def _emit_args(em: _Emitter) -> None:
_emit_utils_minimal(em)
em.line('__ct_args=("$@")')
for m in ARGS_METHODS.values():
if m.bash_impl:
em.line(f"{m.bash_func} () {{ {_to_ret(m.bash_impl)}; }}")
em.blank()
def _emit_utils_minimal(em: _Emitter) -> None:
em.line(f"{CORE_FUNCTIONS['exit'].bash_func} () {{ {CORE_FUNCTIONS['exit'].bash_impl}; }}")
em.line(f"{CORE_FUNCTIONS['is_number'].bash_func} () {{ {_to_ret(CORE_FUNCTIONS['is_number'].bash_impl)}; }}")
em.line(f"{CORE_FUNCTIONS['is_empty'].bash_func} () {{ {_to_ret(CORE_FUNCTIONS['is_empty'].bash_impl)}; }}")
em.blank()
def _emit_test(em: _Emitter) -> None:
em.line("declare -g __ct_test_passed=0 __ct_test_failed=0 __ct_test_current=''")
em.line("__ct_assert () {")
em.indent()
em.line('local cond="$1" msg="${2:-assertion failed}"')
em.line('[[ "$cond" == "true" ]] && { __ct_test_passed=$((__ct_test_passed+1)); return 0; }')
em.line('echo "FAIL [$__ct_test_current]: $msg"')
em.line('__ct_test_failed=$((__ct_test_failed+1))')
em.dedent()
em.line("}")
em.blank()
def _emit_busing_misc(em: _Emitter) -> None:
pass
compiler/backend/bash/stmt.py 0 → 100644
View file @ 31bfe20c
"""
Statement code generation for BashBackend.
All functions take (node, ctx) where ctx is EmitContext.
They write to ctx via ctx.emit() and ctx.indented().
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from ...ir.nodes import (
IRStmt, IRBlock, IRAssign, IRFieldAssign, IRIndexAssign,
IRExprStmt, IRReturn, IRBreak, IRContinue,
IRIf, IRWhile, IRFor, IRForeach, IRWith,
IRTry, IRThrow, IRDefer, IRAwait, IROnSignal, IRWhen,
IRCall, IRMethodCall, IRNew, IRArray, IRDict,
IRIdentifier, IRThis, IRFieldAccess, IRInt,
IRAsync, IRUserDecoratorCall,
)
from .constants import RET_VAR, RET_ARR, OBJ_STORE, COPROC_PREFIX
from .expr import emit_expr, emit_expr_as_stmt, _expr, _ct_args, _var_name
if TYPE_CHECKING:
from .backend import EmitContext
# ---------------------------------------------------------------------------
# Public entry
# ---------------------------------------------------------------------------
def emit_stmt(node: IRStmt, ctx: 'EmitContext') -> None:
_stmt(node, ctx)
def emit_block(block, ctx: 'EmitContext') -> None:
for s in block.stmts:
_stmt(s, ctx)
# ---------------------------------------------------------------------------
# Dispatcher
# ---------------------------------------------------------------------------
def _stmt(node, ctx: 'EmitContext') -> None:
if isinstance(node, IRAssign):
_assign(node, ctx)
elif isinstance(node, IRFieldAssign):
_field_assign(node, ctx)
elif isinstance(node, IRIndexAssign):
_index_assign(node, ctx)
elif isinstance(node, IRExprStmt):
if node.expr is not None:
emit_expr_as_stmt(node.expr, ctx)
elif isinstance(node, IRReturn):
_return(node, ctx)
elif isinstance(node, IRBreak):
ctx.emit('break')
elif isinstance(node, IRContinue):
ctx.emit('continue')
elif isinstance(node, IRBlock):
emit_block(node, ctx)
elif isinstance(node, IRIf):
_if(node, ctx)
elif isinstance(node, IRWhile):
_while(node, ctx)
elif isinstance(node, IRFor):
_for(node, ctx)
elif isinstance(node, IRForeach):
_foreach(node, ctx)
elif isinstance(node, IRWith):
_with(node, ctx)
elif isinstance(node, IRTry):
_try(node, ctx)
elif isinstance(node, IRThrow):
_throw(node, ctx)
elif isinstance(node, IRDefer):
_defer(node, ctx)
elif isinstance(node, IRAwait):
_await(node, ctx)
elif isinstance(node, IROnSignal):
_on_signal(node, ctx)
elif isinstance(node, IRWhen):
_when(node, ctx)
elif isinstance(node, IRUserDecoratorCall):
_user_decorator_call(node, ctx)
# ---------------------------------------------------------------------------
# Special namespace-call helpers for assignment context
# ---------------------------------------------------------------------------
def _is_ns_method(value, ns: str, method: str) -> bool:
return (
isinstance(value, IRMethodCall)
and value.kind == 'stdlib'
and isinstance(value.receiver, IRIdentifier)
and value.receiver.name == ns
and value.method_name == method
)
def _assign_json_parse(node: IRAssign, ctx: 'EmitContext') -> bool:
if not _is_ns_method(node.value, 'json', 'parse'):
return False
target = node.target
json_arg = _expr(node.value.args[0], ctx) if node.value.args else '""'
if node.is_local:
ctx.emit(f'local -A {target}=()')
ctx.emit(f'__ct_json_parse {json_arg} "{target}"')
ctx.dict_vars.add(target)
if node.is_local:
ctx.declare_local(target)
return True
def _assign_json_unmarshal(node: IRAssign, ctx: 'EmitContext') -> bool:
if not _is_ns_method(node.value, 'json', 'unmarshal'):
return False
target = node.target
args = node.value.args
json_arg = _expr(args[0], ctx) if args else '""'
class_name = args[1].name if len(args) > 1 and isinstance(args[1], IRIdentifier) else (
_expr(args[1], ctx).strip('"') if len(args) > 1 else ''
)
ctx.emit(f'__ct_json_unmarshal {json_arg} "{class_name}"')
ctx.emit(f'{target}="$__ct_last_instance"')
return True
def _assign_reflect_fields(node: IRAssign, ctx: 'EmitContext') -> bool:
if not _is_ns_method(node.value, 'reflect', 'fields'):
return False
target = node.target
obj_arg = _expr(node.value.args[0], ctx) if node.value.args else '""'
ctx.emit(f'__ct_reflect_fields {obj_arg}')
if node.is_local:
ctx.emit(f'local -a {target}=("${{{RET_ARR}[@]}}")')
ctx.declare_local(target)
else:
ctx.emit(f'{target}=("${{{RET_ARR}[@]}}")')
ctx.array_vars.add(target)
return True
def _assign_reflect_create(node: IRAssign, ctx: 'EmitContext') -> bool:
if not _is_ns_method(node.value, 'reflect', 'create'):
return False
target = node.target
class_arg = _expr(node.value.args[0], ctx) if node.value.args else '""'
ctx.emit(f'__ct_reflect_create {class_arg}')
ctx.emit(f'{target}="$__ct_last_instance"')
return True
# ---------------------------------------------------------------------------
# Async coproc assignment
# ---------------------------------------------------------------------------
def _assign_async(node: IRAssign, ctx: 'EmitContext') -> bool:
"""Handle `target = async cmd(args)` — emit coproc setup and track handle."""
if not isinstance(node.value, IRAsync):
return False
inner = node.value.expr
if not isinstance(inner, IRCall):
return False
from .expr import _shell_args, _ct_args
cp = ctx.fresh_coproc()
target = node.target
if inner.is_shell_cmd:
cmd = inner.callee_name
args = _shell_args(inner.args, ctx)
cmd_str = f'{cmd} {args}' if args else cmd
else:
cmd = inner.callee.bash_name() if inner.callee else inner.callee_name
args = _ct_args(inner.args, ctx)
cmd_str = f'{cmd} {args}' if args else cmd
ctx.emit(f'coproc {cp} {{ exec {cmd_str}; }}')
ctx.emit(f'exec {{{cp}_wr}}>&${{{cp}[1]}}')
ctx.emit(f'exec {{{cp}_rd}}<&${{{cp}[0]}}')
ctx.emit(f'{cp}_pid=${{{cp}_PID}}')
ctx.emit(f'eval "exec ${{{cp}[1]}}>&-"')
if node.is_local:
ctx.emit(f'local {target}="{cp}"')
ctx.declare_local(target)
else:
ctx.emit(f'{target}="{cp}"')
ctx.process_handles[target] = cp
return True
# ---------------------------------------------------------------------------
# Assignment
# ---------------------------------------------------------------------------
def _assign(node: IRAssign, ctx: 'EmitContext') -> None:
target = node.target
value = node.value
op = node.op
if value is None:
if node.is_local:
ctx.emit(f'local {target}=""')
return
# Array literal → local -a / declare -A
if isinstance(value, IRArray):
elems = ' '.join(_expr(e, ctx) for e in value.elements)
if node.is_local:
ctx.emit(f'local -a {target}=({elems})')
else:
ctx.emit(f'{target}=({elems})')
ctx.declare_local(target) if node.is_local else None
ctx.array_vars.add(target)
return
if isinstance(value, IRDict):
pairs = ' '.join(
f'[{_expr(k, ctx)}]={_expr(v, ctx)}'
for k, v in value.pairs
)
if node.is_local:
ctx.emit(f'local -A {target}=({pairs})')
else:
ctx.emit(f'declare -A {target}=({pairs})')
ctx.dict_vars.add(target)
return
# env.VAR = value → export
if node.is_export:
val = _expr(value, ctx)
ctx.emit(f'export {target}={val}')
return
# Augmented assignment with arithmetic
if op in ('+=', '-=', '*=', '/=', '%='):
val = _expr(value, ctx)
ctx.emit(f'(( {target} {op} {val} ))')
return
# async expr → coproc setup
if _assign_async(node, ctx):
return
# Special namespace calls that need custom assignment logic
if _assign_json_parse(node, ctx):
return
if _assign_json_unmarshal(node, ctx):
return
if _assign_reflect_fields(node, ctx):
return
if _assign_reflect_create(node, ctx):
return
val = _expr(value, ctx)
# Method calls that use __CT_RET_ARR (array return)
from .expr import _ARR_RETURN_METHODS, _STDLIB_METHOD_PREFIX
if isinstance(value, IRMethodCall) and value.method_name in _ARR_RETURN_METHODS:
if node.is_local:
ctx.emit(f'local -a {target}=("${{{RET_ARR}[@]}}")')
else:
ctx.emit(f'{target}=("${{{RET_ARR}[@]}}")')
ctx.array_vars.add(target)
return
# Value already contains the result (emit() was called inside _expr)
if node.is_local:
if ctx.is_declared(target):
ctx.emit(f'{target}={val}')
else:
ctx.emit(f'local {target}={val}')
ctx.declare_local(target)
else:
ctx.emit(f'{target}={val}')
def _field_assign(node: IRFieldAssign, ctx: 'EmitContext') -> None:
val = _expr(node.value, ctx) if node.value else '""'
op = node.op
if isinstance(node.receiver, IRThis):
key = f'$__ct_this.{node.field_name}'
if op in ('+=', '-=', '*=', '/='):
existing = f'"${{{OBJ_STORE}["$__ct_this.{node.field_name}"]}}"'
ctx.emit(f'{OBJ_STORE}["$__ct_this.{node.field_name}"]="$(( {existing} {op[0]} {val} ))"')
else:
ctx.emit(f'{OBJ_STORE}["$__ct_this.{node.field_name}"]={val}')
return
recv = _expr(node.receiver, ctx)
recv_name = recv.strip('"').strip('${}')
# Dict-as-struct: if receiver is a known dict var, use dict subscript syntax
if recv_name in ctx.dict_vars or node.receiver.type.kind == 'dict':
if op in ('+=', '-=', '*=', '/='):
existing = f'"${{{recv_name}["{node.field_name}"]}}"'
ctx.emit(f'{recv_name}["{node.field_name}"]="$(( {existing} {op[0]} {val} ))"')
else:
ctx.emit(f'{recv_name}["{node.field_name}"]={val}')
return
if op in ('+=', '-=', '*=', '/='):
existing = f'"${{{OBJ_STORE}["${{{recv_name}}}.{node.field_name}"]}}"'
ctx.emit(f'{OBJ_STORE}["${{{recv_name}}}.{node.field_name}"]="$(( {existing} {op[0]} {val} ))"')
else:
ctx.emit(f'{OBJ_STORE}["${{{recv_name}}}.{node.field_name}"]={val}')
def _index_assign(node: IRIndexAssign, ctx: 'EmitContext') -> None:
obj = _expr(node.object, ctx)
idx = _expr(node.index, ctx)
val = _expr(node.value, ctx) if node.value else '""'
obj_name = obj.strip('"').strip('${}')
ctx.emit(f'{obj_name}[{idx}]={val}')
# ---------------------------------------------------------------------------
# Return
# ---------------------------------------------------------------------------
def _return(node: IRReturn, ctx: 'EmitContext') -> None:
if node.value is None:
ctx.emit('return 0')
return
from .expr import _ARR_RETURN_METHODS
if isinstance(node.value, IRArray):
elems = ' '.join(_expr(e, ctx) for e in node.value.elements)
ctx.emit(f'{RET_ARR}=({elems})')
ctx.emit('return 0')
return
val = _expr(node.value, ctx)
if isinstance(node.value, IRMethodCall) and node.value.method_name in _ARR_RETURN_METHODS:
ctx.emit(f'{RET_ARR}=("${{{RET_ARR}[@]}}")')
ctx.emit('return 0')
return
ctx.emit(f'{RET_VAR}={val}')
ctx.emit('return 0')
# ---------------------------------------------------------------------------
# Control flow
# ---------------------------------------------------------------------------
_BASH_CMP_NUM = {'==': '-eq', '!=': '-ne', '<': '-lt', '>': '-gt', '<=': '-le', '>=': '-ge'}
def _condition_bash(cond, ctx: 'EmitContext') -> str:
"""Generate a bash condition directly from IR node (for if/while/elif)."""
from .expr import _expr as expr_, _to_arith, _var_name
from ...ir.nodes import IRBinaryOp, IRUnaryOp, IRBool, IRIdentifier
# Literal booleans
if isinstance(cond, IRBool):
return 'true' if cond.value else 'false'
# Direct comparison — generate [[ ]] or (( )) directly
if isinstance(cond, IRBinaryOp):
op = cond.operator
lt = cond.left.type
rt = cond.right.type
if op in ('==', '!=', '<', '>', '<=', '>='):
lv = expr_(cond.left, ctx)
rv = expr_(cond.right, ctx)
# Numeric comparison
if lt.kind in ('int', 'float') or rt.kind in ('int', 'float'):
la = _to_arith(lv)
ra = _to_arith(rv)
bash_op = _BASH_CMP_NUM[op]
return f'[[ "{la}" {bash_op} "{ra}" ]]'
# String comparison
str_op = {'==': '==', '!=': '!=', '<': '<', '>': '>', '<=': '<=', '>=': '>='}.get(op, op)
return f'[[ {lv} {str_op} {rv} ]]'
if op == '&&':
lc = _condition_bash(cond.left, ctx)
rc = _condition_bash(cond.right, ctx)
return f'{lc} && {rc}'
if op == '||':
lc = _condition_bash(cond.left, ctx)
rc = _condition_bash(cond.right, ctx)
return f'{lc} || {rc}'
# Logical negation
if isinstance(cond, IRUnaryOp) and cond.operator == '!':
inner = _condition_bash(cond.operand, ctx)
return f'! ( {inner} )'
# Variable or call that returns "true"/"false"
bash = expr_(cond, ctx)
if bash == '"true"':
return 'true'
if bash == '"false"':
return 'false'
return f'[[ {bash} == "true" ]]'
def _if(node: IRIf, ctx: 'EmitContext') -> None:
cond = _condition_bash(node.condition, ctx)
ctx.emit(f'if {cond}; then')
with ctx.indented():
if node.then_block:
emit_block(node.then_block, ctx)
else:
ctx.emit(':')
_emit_elif_else(node.elif_branches, node.else_block, ctx)
ctx.emit('fi')
def _emit_elif_else(elifs: list, else_block, ctx: 'EmitContext',
extra_fi: int = 0) -> None:
for i, (elif_cond, elif_block) in enumerate(elifs):
saved_len = len(ctx._output)
ec = _condition_bash(elif_cond, ctx)
if len(ctx._output) > saved_len:
side_lines = ctx._output[saved_len:]
del ctx._output[saved_len:]
ctx.emit('else')
with ctx.indented():
for raw in side_lines:
ctx._output.append(' ' + raw)
ctx.emit(f'if {ec}; then')
with ctx.indented():
emit_block(elif_block, ctx)
_emit_elif_else(elifs[i + 1:], else_block, ctx)
ctx.emit('fi')
return
else:
ctx.emit(f'elif {ec}; then')
with ctx.indented():
emit_block(elif_block, ctx)
if else_block:
ctx.emit('else')
with ctx.indented():
emit_block(else_block, ctx)
def _while(node: IRWhile, ctx: 'EmitContext') -> None:
cond = _condition_bash(node.condition, ctx)
ctx.emit(f'while {cond}; do')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('done')
def _for(node: IRFor, ctx: 'EmitContext') -> None:
"""for i in range(...)"""
from ...ir.nodes import IRCall
from .expr import _BUILTIN_BASH
if isinstance(node.iterable, IRCall):
bn = _BUILTIN_BASH.get(node.iterable.callee_name, node.iterable.callee_name)
args = _ct_args(node.iterable.args, ctx)
ctx.emit(f'for {node.variable} in $({bn} {args}); do')
else:
iterable = _expr(node.iterable, ctx) if node.iterable else '"$@"'
ctx.emit(f'for {node.variable} in {iterable}; do')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('done')
def _foreach(node: IRForeach, ctx: 'EmitContext') -> None:
vars_ = node.variables
iterable = node.iterable
kind = node.iter_kind
if kind == 'process':
# foreach line in proc — read from coproc fd
proc = _expr(iterable, ctx)
proc_name = proc.strip('"').strip('${}')
cp = ctx.process_handles.get(proc_name)
fd = f'${cp}_rd' if cp else f'${{{proc_name}[out]}}'
var = vars_[0] if vars_ else '__ct_item'
ctx.emit(f'while IFS= read -r {var} <&{fd}; do')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('done')
return
if kind == 'dict':
d = _expr(iterable, ctx)
d_name = d.strip('"').strip('${}')
if len(vars_) == 2:
kvar, vvar = vars_
ctx.emit(f'for {kvar} in "${{!{d_name}[@]}}"; do')
with ctx.indented():
decl = 'local ' if ctx.in_function else ''
ctx.emit(f'{decl}{vvar}="${{{d_name}[${{{kvar}}}]}}"')
if node.body:
emit_block(node.body, ctx)
else:
var = vars_[0]
ctx.emit(f'for {var} in "${{!{d_name}[@]}}"; do')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('done')
return
# range() call → iterate over stdout sequence
from ...ir.nodes import IRCall as _IRCall
if isinstance(iterable, _IRCall):
bash_name = iterable.callee_name
from .expr import _BUILTIN_BASH
bash_name = _BUILTIN_BASH.get(bash_name, bash_name)
args = _ct_args(iterable.args, ctx)
var = vars_[0] if vars_ else '__ct_item'
ctx.emit(f'for {var} in $({bash_name} {args}); do')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('done')
return
# array: iterate via ${arr[@]}
var = vars_[0] if vars_ else '__ct_item'
# Array literal → spill to temp variable first
if isinstance(iterable, IRArray):
tmp = ctx.fresh_tmp()
elems = ' '.join(_expr(e, ctx) for e in iterable.elements)
if ctx.in_function:
ctx.emit(f'local -a {tmp}=({elems})')
else:
ctx.emit(f'{tmp}=({elems})')
ctx.emit(f'for {var} in "${{{tmp}[@]}}"; do')
else:
from ...ir.nodes import IRIdentifier as _IRIdent
arr = _expr(iterable, ctx)
if '[@]' in arr:
# Already an array expansion (e.g. from split/map/filter) — use directly
ctx.emit(f'for {var} in {arr}; do')
elif isinstance(iterable, _IRIdent) and _var_name(iterable) in ctx.param_array_vars:
# Param holding array name — need nameref to iterate
arr_name = arr.strip('"').strip('${}')
tmp = ctx.fresh_tmp()
decl = 'local' if ctx.in_function else 'declare'
ctx.emit(f'{decl} -n {tmp}="${{{arr_name}}}"')
ctx.emit(f'for {var} in "${{{tmp}[@]}}"; do')
else:
arr_name = arr.strip('"').strip('${}')
ctx.emit(f'for {var} in "${{{arr_name}[@]}}"; do')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('done')
def _with(node: IRWith, ctx: 'EmitContext') -> None:
for var, res in zip(node.variables, node.resources):
val = _expr(res, ctx)
if ctx.in_function:
ctx.emit(f'local {var}={val}')
else:
ctx.emit(f'{var}={val}')
if node.body:
emit_block(node.body, ctx)
# cleanup: call __exit__ on resources
for var in node.variables:
ctx.emit(f'__ct_fh___exit__ "${{{var}}}" 2>/dev/null || true')
def _try(node: IRTry, ctx: 'EmitContext') -> None:
# Use an inline function so that declare -g in __ct_throw propagates out
try_fn = ctx.fresh_tmp() + '_try'
ctx.emit(f'{try_fn} () {{')
with ctx.indented():
if node.try_block:
emit_block(node.try_block, ctx)
ctx.emit(f'}}')
ctx.emit(f'{try_fn} || {{')
with ctx.indented():
if node.except_clauses:
for exc_type, exc_var, exc_block in node.except_clauses:
if exc_var:
decl = 'local' if ctx.in_function else ''
ctx.emit(f'{decl} {exc_var}="${{__ct_exception:-}}"'.strip())
emit_block(exc_block, ctx)
else:
ctx.emit(':')
ctx.emit('}')
ctx.emit(f'unset -f {try_fn}')
if node.finally_block:
emit_block(node.finally_block, ctx)
def _throw(node: IRThrow, ctx: 'EmitContext') -> None:
if node.expr:
val = _expr(node.expr, ctx)
ctx.emit(f'__ct_throw "Error" {val}')
else:
ctx.emit('__ct_throw "Error" ""')
def _defer(node: IRDefer, ctx: 'EmitContext') -> None:
if not node.expr:
return
# Capture the bash lines for the deferred expression
defer_lines: list[str] = []
defer_ctx = ctx.child_context(defer_lines)
emit_expr_as_stmt(node.expr, defer_ctx)
if not defer_lines:
return
body = '; '.join(l.strip() for l in defer_lines if l.strip())
# Escape single quotes for trap
body_esc = body.replace("'", "'\\''")
signal = 'RETURN' if ctx.in_function else 'EXIT'
ctx.emit(f"trap '{body_esc}' {signal}")
def _await(node: IRAwait, ctx: 'EmitContext') -> None:
if node.expr:
proc = _expr(node.expr, ctx)
proc_name = proc.strip('"').strip('${}')
cp = ctx.process_handles.get(proc_name)
if cp:
ctx.emit(f'exec {{{cp}_wr}}>&- 2>/dev/null || :')
ctx.emit(f'exec {{{cp}_rd}}<&- 2>/dev/null || :')
ctx.emit(f'wait ${cp}_pid 2>/dev/null || true')
else:
ctx.emit(f'wait "${{{proc_name}[pid]}}" 2>/dev/null || true')
else:
ctx.emit('wait')
def _on_signal(node: IROnSignal, ctx: 'EmitContext') -> None:
_SIGNAL_SHORT = {
'SIGINT': 'INT', 'SIGTERM': 'TERM', 'SIGHUP': 'HUP',
'SIGUSR1': 'USR1', 'SIGUSR2': 'USR2', 'EXIT': 'EXIT',
}
sig_short = _SIGNAL_SHORT.get(node.signal, node.signal)
handler = f'__ct_on_{sig_short.lower()}'
ctx.emit(f'{handler} () {{')
with ctx.indented():
if node.body:
emit_block(node.body, ctx)
ctx.emit('}')
ctx.emit(f'trap {handler} {sig_short}')
def _when(node: IRWhen, ctx: 'EmitContext') -> None:
val = _expr(node.value, ctx) if node.value else '""'
tmp = ctx.fresh_tmp()
decl = 'local ' if ctx.in_function else ''
ctx.emit(f'{decl}{tmp}={val}')
first = True
for branch in node.branches:
if branch.is_else:
ctx.emit('else')
with ctx.indented():
if branch.body:
emit_block(branch.body, ctx)
else:
parts = []
for p in branch.patterns:
pv = _expr(p, ctx)
parts.append(f'[[ "${{{tmp}}}" == {pv} ]]')
for (lo, hi) in branch.ranges:
lv = _expr(lo, ctx).strip('"')
hv = _expr(hi, ctx).strip('"')
parts.append(f'[[ "${{{tmp}}}" -ge {lv} && "${{{tmp}}}" -le {hv} ]]')
if not parts:
parts = ['false']
cond = ' || '.join(parts)
kw = 'if' if first else 'elif'
ctx.emit(f'{kw} {cond}; then')
with ctx.indented():
if branch.body:
emit_block(branch.body, ctx)
first = False
ctx.emit('fi')
def _user_decorator_call(node: 'IRUserDecoratorCall', ctx: 'EmitContext') -> None:
"""Emit user decorator registration: __ct_call_method "$obj" "method" args "fn_name"."""
args_bash = ' '.join(_expr(a, ctx) for a in node.args)
sep = ' ' if args_bash else ''
ctx.emit(
f'__ct_call_method "${{{node.obj_name}}}" "{node.method_name}"'
f'{sep}{args_bash} "{node.fn_bash_name}"'
)
bootstrap/main.py → compiler/cli.py
View file @ 31bfe20c
...@@ -5,82 +5,89 @@ import tempfile ...@@ -5,82 +5,89 @@ import tempfile
import subprocess import subprocess
from pathlib import Path from pathlib import Path
from .lexer import Lexer from .syntax.parser import parse
from .parser import Parser from .semantics.resolver import resolve
from .codegen import CodeGenerator from .semantics.checker import check_types
from .ir.builder import build
from .optimizer.dce import run_dce
def parse_file (source_path: str): from .optimizer.fold import run_fold
"""Parse a .ct file and return AST or None on error.""" from .backend.bash import compile_to_bash
from .symbols.table import SymbolTable
def compile_source (source: str, filename: str = '<stdin>',
type_check: bool = True, warn_types: bool = False) -> tuple[bool, str, list]:
ast, parse_errs = parse (source, filename)
if parse_errs:
return False, "", parse_errs
scope, cg, resolve_errs = resolve (ast, filename)
if resolve_errs:
return False, "", resolve_errs
if type_check:
type_diags = check_types (None, scope)
if type_diags and not warn_types:
return False, "", type_diags
if type_diags and warn_types:
for d in type_diags:
print (f"Warning: {d.message}", file=sys.stderr)
ir, build_errs = build (ast, scope, cg, filename)
if build_errs:
return False, "", build_errs
ir = run_fold (ir)
ir, used_cats = run_dce (ir, cg)
script = compile_to_bash (ir, used_cats)
return True, script, []
def compile_source_with_symbols (source: str, filename: str = '<stdin>') -> tuple[bool, str, list, 'SymbolTable | None']:
ast, parse_errs = parse (source, filename)
if parse_errs:
return False, "", parse_errs, None
scope, cg, resolve_errs = resolve (ast, filename)
if resolve_errs:
return False, "", resolve_errs, None
table = SymbolTable (root_scope=scope, call_graph=cg, filename=filename)
ir, build_errs = build (ast, scope, cg, filename)
if build_errs:
return False, "", build_errs, None
ir = run_fold (ir)
ir, used_cats = run_dce (ir, cg)
script = compile_to_bash (ir, used_cats)
return True, script, [], table
def compile_files (source_paths: list, type_check: bool = True,
warn_types: bool = False) -> tuple[bool, str]:
combined_source = ""
for sp in source_paths:
try: try:
with open (source_path, "r", encoding="utf-8") as f: with open (sp, "r", encoding="utf-8") as f:
source = f.read () combined_source += f.read () + "\n"
except FileNotFoundError: except FileNotFoundError:
print (f"Error: File not found: {source_path}", file=sys.stderr) print (f"Error: File not found: {sp}", file=sys.stderr)
return None return False, ""
except Exception as e: except Exception as e:
print (f"Error reading file: {e}", file=sys.stderr) print (f"Error reading file: {e}", file=sys.stderr)
return None
filename = os.path.basename (source_path)
lexer = Lexer (source, filename)
tokens = lexer.tokenize ()
if lexer.errors:
for error in lexer.errors:
print (str (error), file=sys.stderr)
return None
parser = Parser (tokens, filename)
ast = parser.parse ()
if parser.errors.has_errors ():
parser.errors.print_errors ()
return None
return ast
def compile_file (source_path: str, output_path: str = None) -> tuple[bool, str]:
"""Compile a single .ct file to bash."""
return compile_files ([source_path])
def _get_pkgdatadir () -> str:
"""Get package data directory (for meson-installed libs)."""
bootstrap_dir = os.path.dirname (os.path.abspath (__file__))
project_root = os.path.dirname (bootstrap_dir)
return os.path.join (project_root, "lib")
def compile_files (source_paths: list, type_check: bool = True, warn_types: bool = False) -> tuple[bool, str]:
"""Compile multiple .ct files to bash (Vala-style multi-file)."""
asts = []
parsed_files = set ()
for source_path in source_paths:
abs_path = os.path.abspath (source_path)
if abs_path in parsed_files:
continue
parsed_files.add (abs_path)
ast = parse_file (source_path)
if ast is None:
return False, "" return False, ""
asts.append (ast)
codegen = CodeGenerator (type_check=type_check, warn_types=warn_types) fname = os.path.basename (source_paths[0]) if source_paths else '<stdin>'
output = codegen.generate_multi (asts) ok, script, errs = compile_source (combined_source, fname,
type_check=type_check, warn_types=warn_types)
if codegen.errors.has_errors (): if not ok:
codegen.errors.print_errors () for e in errs:
return False, "" print (str (e), file=sys.stderr)
return ok, script
return True, output
def find_ct_files (directory: str = ".") -> list: def find_ct_files (directory: str = ".") -> list:
"""Find all .ct files in directory recursively, sorted (namespace files first, main.ct last)."""
import glob import glob
files = sorted (glob.glob (os.path.join (directory, "**", "*.ct"), recursive=True)) files = sorted (glob.glob (os.path.join (directory, "**", "*.ct"), recursive=True))
files += [f for f in sorted (glob.glob (os.path.join (directory, "*.ct"))) if f not in files] files += [f for f in sorted (glob.glob (os.path.join (directory, "*.ct"))) if f not in files]
...@@ -99,20 +106,24 @@ def find_ct_files (directory: str = ".") -> list: ...@@ -99,20 +106,24 @@ def find_ct_files (directory: str = ".") -> list:
return ns_files + other_files + main_files return ns_files + other_files + main_files
def cmd_build (args): def _resolve_sources (source_paths):
"""Build command - compile .ct to .sh (supports multiple files)"""
source_paths = args.sources
if len (source_paths) == 1 and os.path.isdir (source_paths[0]): if len (source_paths) == 1 and os.path.isdir (source_paths[0]):
directory = source_paths[0] directory = source_paths[0]
source_paths = find_ct_files (directory) source_paths = find_ct_files (directory)
if not source_paths: if not source_paths:
print (f"Error: No .ct files found in {directory}", file=sys.stderr) print (f"Error: No .ct files found in {directory}", file=sys.stderr)
return 1 return None
else: else:
for source_path in source_paths: for source_path in source_paths:
if not source_path.endswith (".ct"): if not source_path.endswith (".ct"):
print (f"Error: Source file must have .ct extension: {source_path}", file=sys.stderr) print (f"Error: Source file must have .ct extension: {source_path}", file=sys.stderr)
return None
return source_paths
def cmd_build (args):
source_paths = _resolve_sources (args.sources)
if source_paths is None:
return 1 return 1
if args.output: if args.output:
...@@ -120,15 +131,34 @@ def cmd_build (args): ...@@ -120,15 +131,34 @@ def cmd_build (args):
else: else:
output_path = source_paths[0].replace (".ct", ".sh") output_path = source_paths[0].replace (".ct", ".sh")
type_check = not getattr (args, 'no_type_check', False) save_symbols = getattr (args, 'save_symbols', None)
warn_types = getattr (args, 'warn_types', False)
success, output = compile_files (source_paths, type_check=type_check, warn_types=warn_types) if save_symbols:
if not success: combined = ""
for sp in source_paths:
with open (sp, "r", encoding="utf-8") as f:
combined += f.read () + "\n"
fname = os.path.basename (source_paths[0])
ok, script, errs, table = compile_source_with_symbols (combined, fname)
if not ok:
for e in errs:
print (str (e), file=sys.stderr)
return 1
sym_dir = os.path.dirname (save_symbols)
if sym_dir:
os.makedirs (sym_dir, exist_ok=True)
table.save (save_symbols)
print (f"Symbols saved: {save_symbols} ({len (table.all_symbols ())} symbols)")
else:
tc = not getattr (args, 'no_type_check', False)
wt = getattr (args, 'warn_types', False)
ok, script = compile_files (source_paths, type_check=tc, warn_types=wt)
if not ok:
return 1 return 1
try: try:
with open (output_path, "w", encoding="utf-8") as f: with open (output_path, "w", encoding="utf-8") as f:
f.write (output) f.write (script)
os.chmod (output_path, 0o755) os.chmod (output_path, 0o755)
if len (source_paths) == 1: if len (source_paths) == 1:
print (f"Compiled: {source_paths[0]} -> {output_path}") print (f"Compiled: {source_paths[0]} -> {output_path}")
...@@ -155,7 +185,6 @@ def cmd_build (args): ...@@ -155,7 +185,6 @@ def cmd_build (args):
def cmd_run (args): def cmd_run (args):
"""Run command - compile and execute (supports multiple files)"""
all_args = args.sources_and_args all_args = args.sources_and_args
if "--" in all_args: if "--" in all_args:
...@@ -166,20 +195,12 @@ def cmd_run (args): ...@@ -166,20 +195,12 @@ def cmd_run (args):
source_paths = all_args source_paths = all_args
script_args = [] script_args = []
if len (source_paths) == 1 and os.path.isdir (source_paths[0]): source_paths = _resolve_sources (source_paths)
directory = source_paths[0] if source_paths is None:
source_paths = find_ct_files (directory)
if not source_paths:
print (f"Error: No .ct files found in {directory}", file=sys.stderr)
return 1
else:
for source_path in source_paths:
if not source_path.endswith (".ct"):
print (f"Error: Source file must have .ct extension: {source_path}", file=sys.stderr)
return 1 return 1
success, output = compile_files (source_paths) ok, script = compile_files (source_paths)
if not success: if not ok:
return 1 return 1
try: try:
...@@ -189,7 +210,7 @@ def cmd_run (args): ...@@ -189,7 +210,7 @@ def cmd_run (args):
delete=False, delete=False,
encoding="utf-8" encoding="utf-8"
) as f: ) as f:
f.write (output) f.write (script)
temp_path = f.name temp_path = f.name
os.chmod (temp_path, 0o755) os.chmod (temp_path, 0o755)
...@@ -212,39 +233,39 @@ def cmd_run (args): ...@@ -212,39 +233,39 @@ def cmd_run (args):
def cmd_test (args): def cmd_test (args):
"""Test command - run @test functions""" source_paths = _resolve_sources (args.sources)
source_paths = args.sources if source_paths is None:
if len (source_paths) == 1 and os.path.isdir (source_paths[0]):
directory = source_paths[0]
source_paths = find_ct_files (directory)
if not source_paths:
print (f"Error: No .ct files found in {directory}", file=sys.stderr)
return 1
else:
for source_path in source_paths:
if not source_path.endswith (".ct"):
print (f"Error: Source file must have .ct extension: {source_path}", file=sys.stderr)
return 1 return 1
asts = [] combined = ""
for source_path in source_paths: for sp in source_paths:
ast = parse_file (source_path) with open (sp, "r", encoding="utf-8") as f:
if ast is None: combined += f.read () + "\n"
return 1
asts.append (ast)
codegen = CodeGenerator () fname = os.path.basename (source_paths[0])
output = codegen.generate_multi (asts, test_mode=True)
if codegen.errors.has_errors (): ast, parse_errs = parse (combined, fname)
codegen.errors.print_errors () if parse_errs:
for e in parse_errs:
print (str (e), file=sys.stderr)
return 1 return 1
if not codegen.test_functions: scope, cg, resolve_errs = resolve (ast, fname)
print ("No @test functions found", file=sys.stderr) if resolve_errs:
for e in resolve_errs:
print (str (e), file=sys.stderr)
return 1 return 1
ir, build_errs = build (ast, scope, cg, fname)
if build_errs:
for e in build_errs:
print (str (e), file=sys.stderr)
return 1
ir = run_fold (ir)
ir, used_cats = run_dce (ir, cg, keep_tests=True)
script = compile_to_bash (ir, used_cats)
try: try:
with tempfile.NamedTemporaryFile ( with tempfile.NamedTemporaryFile (
mode="w", mode="w",
...@@ -252,7 +273,7 @@ def cmd_test (args): ...@@ -252,7 +273,7 @@ def cmd_test (args):
delete=False, delete=False,
encoding="utf-8" encoding="utf-8"
) as f: ) as f:
f.write (output) f.write (script)
temp_path = f.name temp_path = f.name
os.chmod (temp_path, 0o755) os.chmod (temp_path, 0o755)
...@@ -275,7 +296,6 @@ def cmd_test (args): ...@@ -275,7 +296,6 @@ def cmd_test (args):
def cmd_build_lib (args): def cmd_build_lib (args):
"""Build library command"""
source = args.source source = args.source
output_path = getattr (args, 'output', None) output_path = getattr (args, 'output', None)
install = getattr (args, 'install', False) install = getattr (args, 'install', False)
...@@ -297,22 +317,44 @@ def cmd_build_lib (args): ...@@ -297,22 +317,44 @@ def cmd_build_lib (args):
if not output_path: if not output_path:
output_path = source.replace (".ct", ".sh") output_path = source.replace (".ct", ".sh")
asts = [] combined = ""
for source_path in source_paths: for sp in source_paths:
ast = parse_file (source_path) try:
if ast is None: with open (sp, "r", encoding="utf-8") as f:
combined += f.read () + "\n"
except Exception as e:
print (f"Error: {e}", file=sys.stderr)
return 1 return 1
asts.append (ast)
codegen = CodeGenerator () fname = os.path.basename (source_paths[0])
code = codegen.generate_multi (asts) ast, parse_errs = parse (combined, fname)
if parse_errs:
for e in parse_errs:
print (str (e), file=sys.stderr)
return 1
scope, cg, resolve_errs = resolve (ast, fname)
if resolve_errs:
for e in resolve_errs:
print (str (e), file=sys.stderr)
return 1
if codegen.errors.has_errors (): ir, build_errs = build (ast, scope, cg, fname)
codegen.errors.print_errors () if build_errs:
for e in build_errs:
print (str (e), file=sys.stderr)
return 1 return 1
ns_list = " ".join (sorted (codegen.namespaces.keys ())) if codegen.namespaces else lib_name ir = run_fold (ir)
metadata = f"# content-lib: {lib_name}\n# content-version: 1.0.0\n# content-namespaces: {ns_list}\n" namespaces = sorted ({fn.symbol.scope.name for fn in ir.functions
if fn.symbol and fn.symbol.scope and fn.symbol.scope.kind == 'namespace'})
all_cats = set ()
for fn in ir.functions:
all_cats.add ('core')
code = compile_to_bash (ir, all_cats if all_cats else {'core'})
ns_str = ",".join (namespaces) if namespaces else "none"
metadata = f"# content-lib: {lib_name}\n# content-version: 1.0.0\n# content-namespaces: {ns_str}\n"
output = metadata + code output = metadata + code
try: try:
...@@ -344,7 +386,7 @@ def main (): ...@@ -344,7 +386,7 @@ def main ():
parser.add_argument ( parser.add_argument (
"--version", "-v", "--version", "-v",
action="version", action="version",
version="ContenT 0.1.0 (bootstrap)" version="ContenT 0.1.0"
) )
subparsers = parser.add_subparsers (dest="command", help="Commands") subparsers = parser.add_subparsers (dest="command", help="Commands")
...@@ -353,8 +395,9 @@ def main (): ...@@ -353,8 +395,9 @@ def main ():
build_parser.add_argument ("sources", nargs="+", help="Source files (.ct)") build_parser.add_argument ("sources", nargs="+", help="Source files (.ct)")
build_parser.add_argument ("-o", "--output", help="Output file (.sh)") build_parser.add_argument ("-o", "--output", help="Output file (.sh)")
build_parser.add_argument ("--lint", action="store_true", help="Run ShellCheck") build_parser.add_argument ("--lint", action="store_true", help="Run ShellCheck")
build_parser.add_argument ("--save-symbols", metavar="PATH", help="Save symbol table to JSON (for LSP)")
build_parser.add_argument ("--no-type-check", action="store_true", help="Disable type checking") build_parser.add_argument ("--no-type-check", action="store_true", help="Disable type checking")
build_parser.add_argument ("--warn-types", action="store_true", help="Show type errors as warnings") build_parser.add_argument ("--warn-types", action="store_true", help="Type errors as warnings")
run_parser = subparsers.add_parser ("run", help="Compile and run (use -- to separate script args)") run_parser = subparsers.add_parser ("run", help="Compile and run (use -- to separate script args)")
run_parser.add_argument ("sources_and_args", nargs="+", help="Source files (.ct) [-- script args]") run_parser.add_argument ("sources_and_args", nargs="+", help="Source files (.ct) [-- script args]")
......
compiler/constants.py 0 → 100644
View file @ 31bfe20c
RET_VAR = "__CT_RET"
RET_ARR = "__CT_RET_ARR"
TMP_PREFIX = "__ct_tmp_"
CLASS_FUNC_PREFIX = "__ct_class_"
LAMBDA_PREFIX = "__ct_lambda_"
OBJ_STORE = "__CT_OBJ"
THIS_VAR = "__ct_this"
ARR_PREFIX = "__ct_arr_"
DICT_PREFIX = "__ct_dict_"
STR_PREFIX = "__ct_str_"
FH_PREFIX = "__ct_fh_"
HTTP_PREFIX = "__ct_http_"
FS_PREFIX = "__ct_fs_"
JSON_PREFIX = "__ct_json_"
REGEX_PREFIX = "__ct_regex_"
MATH_PREFIX = "__ct_math_"
COPROC_PREFIX = "__ct_cp"
compiler/ir/builder.py 0 → 100644
View file @ 31bfe20c
"""
IR Builder — lowers SyntaxTree → IR.
Receives the resolved ScopeTree from Resolver.
All name lookups use the scope tree; no string matching in codegen.
Shell command detection: if a callee name is not resolved to a Symbol
and is not in BUILTIN_FUNCS → IRCall(is_shell_cmd=True).
"""
from __future__ import annotations
from typing import Optional, Union
from ..syntax.nodes import (
Program, FunctionDecl, ClassDecl, ConstructorDecl,
NamespaceDecl, UsingStmt, BusingStmt,
Block, ExpressionStmt, Assignment, ReturnStmt,
BreakStmt, ContinueStmt,
IfStmt, WhileStmt, ForStmt, ForeachStmt, WithStmt,
TryStmt, ThrowStmt, DeferStmt, AwaitStmt, OnSignalStmt, WhenStmt, WhenBranch,
CallExpr, MemberAccess, IndexAccess, BinaryOp, UnaryOp,
Identifier, ThisExpr, NewExpr, AsyncExpr, Lambda, BaseCall,
ArrayLiteral, DictLiteral, StringLiteral, StringInterp, StringText,
IntegerLiteral, FloatLiteral, BoolLiteral, NilLiteral,
Parameter, ClassField, Decorator, RangePattern, ErrorNode,
SyntaxNode, Expression, Statement, Declaration,
UNKNOWN_LOC,
)
from ..semantics.scope import Scope, Symbol, CallGraph
from ..semantics.types import (
ContenTType, T_ANY, T_VOID, T_STRING, T_INT, T_BOOL, T_FLOAT,
array_of, dict_of, class_type, from_annotation, PRIMITIVE_TYPES,
)
from ..semantics.resolver import BUILTIN_FUNCS, STDLIB_NAMESPACES
from ..methods import NAMESPACE_METHODS
from ..methods import (
STRING_METHODS, ARRAY_METHODS, DICT_METHODS,
)
from .nodes import (
IRNode, IRProgram, IRFunction, IRClass, IRClassField, IRParam,
IRDecorator, IRBusingImport, IRUserDecoratorCall,
IRBlock, IRAssign, IRFieldAssign, IRIndexAssign, IRExprStmt,
IRReturn, IRBreak, IRContinue,
IRIf, IRWhile, IRFor, IRForeach, IRWith,
IRTry, IRThrow, IRDefer, IRAwait, IROnSignal, IRWhen, IRWhenBranch,
IRInt, IRFloat, IRBool, IRNil, IRString, IRStringText, IRStringInterp,
IRArray, IRDict, IRIdentifier, IRThis, IRFieldAccess, IRIndexAccess,
IRBinaryOp, IRUnaryOp,
IRCall, IRMethodCall, IRNew, IRAsync, IRLambda,
IRExpr, IRStmt,
)
# Stdlib method namespaces whose .method() calls stay as IRMethodCall(kind='stdlib')
_STDLIB_METHOD_TYPES = frozenset({'string', 'array', 'dict', 'file_handle', 'process'})
# Known stdlib method names — if receiver type is any/unknown and method name matches,
# treat as stdlib (heuristic, avoids needing full type inference for variables)
_STDLIB_METHOD_NAMES = frozenset({
'upper', 'lower', 'trim', 'len', 'contains', 'starts', 'ends',
'replace', 'split', 'substr', 'index', 'charAt', 'urlencode',
'push', 'pop', 'shift', 'join', 'slice', 'map', 'filter',
'has', 'del', 'keys',
'read', 'write', 'writeln', 'readline', 'close',
})
# Names of builtin objects accessed as namespaces (fs.read, http.get, ...)
_STDLIB_NS = STDLIB_NAMESPACES
class IRBuilder:
def __init__(self, scope: Scope, call_graph: CallGraph,
filename: str = '<stdin>'):
self.global_scope = scope
self.call_graph = call_graph
self.filename = filename
self.errors: list[str] = []
def _ct_names(methods_dict):
return frozenset(m.name for m in methods_dict.values())
self._TYPE_METHOD_NAMES: dict[str, frozenset] = {
'string': _ct_names(STRING_METHODS),
'array': _ct_names(ARRAY_METHODS),
'dict': _ct_names(DICT_METHODS),
}
# Resolution context (mirrors Resolver)
self._scope: Scope = scope
self._current_class: Optional[str] = None
self._current_parent_class: Optional[str] = None
self._using_scopes: list[Scope] = []
self._using_aliases: dict[str, str] = {}
self._using_names: dict[str, Symbol] = {}
self._busing: dict[str, str] = {}
self._async_vars: set[str] = set()
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def build(self, program: Program) -> IRProgram:
ir = IRProgram(filename=program.filename or self.filename,
source=program.location or UNKNOWN_LOC)
self._apply_usings(program)
self._build_program(program, ir)
return ir
# ------------------------------------------------------------------
# Top-level
# ------------------------------------------------------------------
def _apply_usings(self, program: Program) -> None:
for stmt in program.statements:
if isinstance(stmt, UsingStmt):
ns_scope = self.global_scope._find_namespace(stmt.namespace)
if stmt.alias:
self._using_aliases[stmt.alias] = stmt.namespace
if stmt.names and ns_scope:
for name in stmt.names:
sym = ns_scope.symbols.get(name)
if sym:
self._using_names[name] = sym
elif ns_scope and not stmt.names:
self._using_scopes.append(ns_scope)
def _build_program(self, program: Program, ir: IRProgram) -> None:
for stmt in program.statements:
if isinstance(stmt, FunctionDecl):
fn = self._build_func(stmt)
ir.functions.append(fn)
# User decorators: generate registration calls at this position
# in the top-level statement sequence (after variables are set up).
for dec in stmt.decorators:
if dec.object:
dec_args = [self._build_expr(v) for _, v in (dec.arguments or [])]
ir.top_stmts.append(IRUserDecoratorCall(
obj_name=dec.object,
method_name=dec.name,
args=dec_args,
fn_bash_name=fn.symbol.bash_name() if fn.symbol else fn.name,
source=dec.location or UNKNOWN_LOC,
))
elif isinstance(stmt, ClassDecl):
ir.classes.append(self._build_class(stmt))
elif isinstance(stmt, NamespaceDecl):
self._build_namespace(stmt, ir)
elif isinstance(stmt, BusingStmt):
ir.busing.append(IRBusingImport(
name=stmt.name, path=stmt.path,
source=stmt.location or UNKNOWN_LOC,
))
if stmt.name:
self._busing[stmt.name] = stmt.path
elif isinstance(stmt, (UsingStmt, ErrorNode)):
pass
else:
ir_stmt = self._build_stmt(stmt)
if ir_stmt:
ir.top_stmts.append(ir_stmt)
def _build_namespace(self, node: NamespaceDecl, ir: IRProgram) -> None:
ns_scope = self.global_scope._find_namespace(node.name)
saved = self._scope
self._scope = ns_scope or saved
for stmt in node.statements:
if isinstance(stmt, FunctionDecl):
fn = self._build_func(stmt, namespace=node.name)
ir.functions.append(fn)
elif isinstance(stmt, ClassDecl):
cl = self._build_class(stmt, namespace=node.name)
ir.classes.append(cl)
self._scope = saved
# ------------------------------------------------------------------
# Function / class builders
# ------------------------------------------------------------------
def _build_func(self, node: FunctionDecl,
namespace: str = '',
class_name: str = '',
is_method: bool = False) -> IRFunction:
sym = self._resolve_name(node.name)
loc = node.location or UNKNOWN_LOC
# Build params
params = [self._build_param(p) for p in node.params]
# Build decorators
decorators = [self._build_decorator(d) for d in node.decorators]
is_awk = any(d.name == 'awk' for d in node.decorators)
is_test = any(d.name == 'test' for d in node.decorators)
# Enter function scope
fn_scope = self._find_child_scope('function', node.name)
saved = self._scope
if fn_scope:
self._scope = fn_scope
body = self._build_block(node.body) if node.body else IRBlock(source=loc)
self._scope = saved
return IRFunction(
symbol=sym, name=node.name,
params=params, body=body,
decorators=decorators,
is_awk=is_awk, is_test=is_test,
is_method=is_method, class_name=class_name,
source=loc,
)
def _build_param(self, p: Parameter) -> IRParam:
typ = from_annotation(p.type_annotation.name,
p.type_annotation.is_array) if p.type_annotation else T_ANY
default = self._build_expr(p.default) if p.default else None
return IRParam(name=p.name, type=typ, default=default,
is_variadic=p.is_variadic,
source=p.location or UNKNOWN_LOC)
def _build_decorator(self, d: Decorator) -> IRDecorator:
args = [(k, self._build_expr(v)) for k, v in (d.arguments or [])]
return IRDecorator(name=d.name, obj_name=d.object, args=args,
source=d.location or UNKNOWN_LOC)
def _build_class(self, node: ClassDecl, namespace: str = '') -> IRClass:
sym = self._resolve_name(node.name)
loc = node.location or UNKNOWN_LOC
saved_class = self._current_class
saved_parent = self._current_parent_class
self._current_class = node.name
self._current_parent_class = node.parent or None
cls_scope = self._find_child_scope('class', node.name)
saved = self._scope
if cls_scope:
self._scope = cls_scope
fields = [self._build_class_field(f) for f in node.fields]
constructor = None
if node.constructor:
constructor = self._build_constructor(node.constructor, node.name)
methods = [
self._build_func(m, class_name=node.name, is_method=True)
for m in node.methods
]
self._scope = saved
self._current_class = saved_class
self._current_parent_class = saved_parent
return IRClass(
symbol=sym, name=node.name, parent=node.parent,
fields=fields, constructor=constructor, methods=methods,
source=loc,
)
def _build_class_field(self, f: ClassField) -> IRClassField:
default = self._build_expr(f.default) if f.default else None
# Infer field_kind from default or annotation
kind = 'scalar'
if isinstance(f.default, ArrayLiteral):
kind = 'array'
elif isinstance(f.default, DictLiteral):
kind = 'dict'
elif isinstance(f.default, NewExpr):
kind = 'object'
elif f.type_annotation:
if f.type_annotation.is_array:
kind = 'array'
elif f.type_annotation.name == 'dict':
kind = 'dict'
typ = from_annotation(f.type_annotation.name,
f.type_annotation.is_array) if f.type_annotation else T_ANY
return IRClassField(name=f.name, type=typ, default=default,
field_kind=kind, source=f.location or UNKNOWN_LOC)
def _build_constructor(self, node: ConstructorDecl,
class_name: str) -> IRFunction:
params = [self._build_param(p) for p in node.params]
ctor_scope = self._find_child_scope('function', 'construct')
saved = self._scope
if ctor_scope:
self._scope = ctor_scope
body = self._build_block(node.body) if node.body else IRBlock()
self._scope = saved
return IRFunction(
name='construct', params=params, body=body,
is_method=True, class_name=class_name,
source=node.location or UNKNOWN_LOC,
)
# ------------------------------------------------------------------
# Statement builders
# ------------------------------------------------------------------
def _build_block(self, node: Block) -> IRBlock:
loc = node.location or UNKNOWN_LOC
stmts = []
for s in (node.statements or []):
ir_s = self._build_stmt(s)
if ir_s:
stmts.append(ir_s)
return IRBlock(stmts=stmts, source=loc)
def _build_stmt(self, node: SyntaxNode) -> Optional[IRStmt]:
loc = getattr(node, 'location', None) or UNKNOWN_LOC
if isinstance(node, FunctionDecl):
# Nested function definition (rare, treat as closure/stub)
return None # handled at program level
if isinstance(node, Assignment):
return self._build_assignment(node)
if isinstance(node, ExpressionStmt):
expr = self._build_expr(node.expression) if node.expression else None
return IRExprStmt(expr=expr, source=loc) if expr else None
if isinstance(node, ReturnStmt):
val = self._build_expr(node.value) if node.value else None
return IRReturn(value=val, source=loc)
if isinstance(node, BreakStmt):
return IRBreak(source=loc)
if isinstance(node, ContinueStmt):
return IRContinue(source=loc)
if isinstance(node, Block):
return self._build_block(node)
if isinstance(node, IfStmt):
cond = self._build_expr(node.condition) if node.condition else IRBool(value=False)
then = self._build_block(node.then_branch) if node.then_branch else IRBlock()
elifs = [
(self._build_expr(c), self._build_block(b))
for c, b in (node.elif_branches or [])
]
else_ = self._build_block(node.else_branch) if node.else_branch else None
return IRIf(condition=cond, then_block=then,
elif_branches=elifs, else_block=else_, source=loc)
if isinstance(node, WhileStmt):
cond = self._build_expr(node.condition) if node.condition else IRBool(value=False)
body = self._build_block(node.body) if node.body else IRBlock()
return IRWhile(condition=cond, body=body, source=loc)
if isinstance(node, ForStmt):
it = self._build_expr(node.iterable) if node.iterable else IRNil()
body = self._build_block(node.body) if node.body else IRBlock()
return IRFor(variable=node.variable, iterable=it, body=body, source=loc)
if isinstance(node, ForeachStmt):
it = self._build_expr(node.iterable) if node.iterable else IRNil()
body = self._build_block(node.body) if node.body else IRBlock()
kind = 'array'
if isinstance(node.iterable, Identifier) and node.iterable.name in self._async_vars:
kind = 'process'
return IRForeach(variables=node.variables, iterable=it, body=body,
iter_kind=kind, source=loc)
if isinstance(node, WithStmt):
resources = [self._build_expr(r) for r in (node.resources or [])]
body = self._build_block(node.body) if node.body else IRBlock()
return IRWith(variables=node.variables, resources=resources,
body=body, source=loc)
if isinstance(node, TryStmt):
try_blk = self._build_block(node.try_block) if node.try_block else IRBlock()
excepts = [
(et, ev, self._build_block(b))
for et, ev, b in (node.except_clauses or [])
]
fin = self._build_block(node.finally_block) if node.finally_block else None
return IRTry(try_block=try_blk, except_clauses=excepts,
finally_block=fin, source=loc)
if isinstance(node, ThrowStmt):
expr = self._build_expr(node.expression) if node.expression else IRNil()
return IRThrow(expr=expr, source=loc)
if isinstance(node, DeferStmt):
expr = self._build_expr(node.expression) if node.expression else IRNil()
return IRDefer(expr=expr, source=loc)
if isinstance(node, AwaitStmt):
expr = self._build_expr(node.expression) if node.expression else IRNil()
return IRAwait(expr=expr, source=loc)
if isinstance(node, OnSignalStmt):
body = self._build_block(node.body) if node.body else IRBlock()
return IROnSignal(signal=node.signal, body=body, source=loc)
if isinstance(node, WhenStmt):
val = self._build_expr(node.value) if node.value else IRNil()
branches = [self._build_when_branch(b) for b in (node.branches or [])]
return IRWhen(value=val, branches=branches, source=loc)
if isinstance(node, (UsingStmt, BusingStmt, NamespaceDecl, ErrorNode)):
return None
return None
def _build_assignment(self, node: Assignment) -> Optional[IRStmt]:
loc = node.location or UNKNOWN_LOC
value = self._build_expr(node.value) if node.value else IRNil()
op = node.operator or '='
target = node.target
if isinstance(target, Identifier):
if isinstance(value, IRAsync):
self._async_vars.add(target.name)
is_env = target.name.startswith('env.')
return IRAssign(target=target.name, value=value, op=op,
is_export=is_env, source=loc)
if isinstance(target, MemberAccess):
# env.VAR = ...
if isinstance(target.object, Identifier) and target.object.name == 'env':
return IRAssign(target=target.member, value=value, op=op,
is_export=True, source=loc)
recv = self._build_expr(target.object) if target.object else IRNil()
return IRFieldAssign(receiver=recv, field_name=target.member,
value=value, op=op, source=loc)
if isinstance(target, IndexAccess):
obj = self._build_expr(target.object) if target.object else IRNil()
idx = self._build_expr(target.index) if target.index else IRNil()
return IRIndexAssign(object=obj, index=idx, value=value, source=loc)
# Fallback
if target:
self.errors.append(f'{loc}: Unsupported assignment target')
return IRAssign(target='__unknown', value=value, op=op, source=loc)
def _build_when_branch(self, b: WhenBranch) -> IRWhenBranch:
loc = b.location or UNKNOWN_LOC
patterns = []
ranges = []
for pat in (b.patterns or []):
if isinstance(pat, RangePattern):
start = self._build_expr(pat.start) if pat.start else IRNil()
end = self._build_expr(pat.end) if pat.end else IRNil()
ranges.append((start, end))
else:
patterns.append(self._build_expr(pat))
body = self._build_block(b.body) if b.body else IRBlock()
return IRWhenBranch(patterns=patterns, is_else=b.is_else,
body=body, ranges=ranges, source=loc)
# ------------------------------------------------------------------
# Expression builders
# ------------------------------------------------------------------
def _build_expr(self, node: Expression) -> IRExpr:
loc = getattr(node, 'location', None) or UNKNOWN_LOC
if isinstance(node, IntegerLiteral):
return IRInt(value=node.value, type=T_INT, source=loc)
if isinstance(node, FloatLiteral):
return IRFloat(value=node.value, type=T_FLOAT, source=loc)
if isinstance(node, BoolLiteral):
return IRBool(value=node.value, type=T_BOOL, source=loc)
if isinstance(node, NilLiteral):
return IRNil(type=T_ANY, source=loc)
if isinstance(node, StringLiteral):
return self._build_string(node)
if isinstance(node, ArrayLiteral):
elems = [self._build_expr(e) for e in node.elements]
return IRArray(elements=elems, type=array_of(T_ANY), source=loc)
if isinstance(node, DictLiteral):
pairs = [(self._build_expr(k), self._build_expr(v)) for k, v in node.pairs]
return IRDict(pairs=pairs, type=dict_of(T_ANY, T_ANY), source=loc)
if isinstance(node, Identifier):
return self._build_identifier(node)
if isinstance(node, ThisExpr):
return IRThis(class_name=self._current_class or '', source=loc)
if isinstance(node, BinaryOp):
left = self._build_expr(node.left) if node.left else IRNil()
right = self._build_expr(node.right) if node.right else IRNil()
op = node.operator
lt, rt = left.type, right.type
if op == '..':
result_type = T_STRING
elif op in ('+', '-', '*', '/', '%', '**'):
if lt.kind == 'string' or rt.kind == 'string':
result_type = T_STRING
elif lt.kind == 'float' or rt.kind == 'float':
result_type = T_FLOAT
elif lt.kind == 'int' and rt.kind == 'int':
result_type = T_INT
else:
result_type = T_ANY
elif op in ('==', '!=', '<', '>', '<=', '>=', '&&', '||'):
result_type = T_BOOL
else:
result_type = T_ANY
return IRBinaryOp(left=left, operator=op, right=right,
type=result_type, source=loc)
if isinstance(node, UnaryOp):
operand = self._build_expr(node.operand) if node.operand else IRNil()
return IRUnaryOp(operator=node.operator, operand=operand,
type=T_ANY, source=loc)
if isinstance(node, CallExpr):
return self._build_call(node)
if isinstance(node, MemberAccess):
return self._build_member_access(node)
if isinstance(node, IndexAccess):
obj = self._build_expr(node.object) if node.object else IRNil()
idx = self._build_expr(node.index) if node.index else IRNil()
return IRIndexAccess(object=obj, index=idx, source=loc)
if isinstance(node, NewExpr):
sym = self._resolve_name(node.class_name)
args = [self._build_expr(a) for a in node.arguments]
return IRNew(class_sym=sym, class_name=node.class_name,
args=args, type=class_type(node.class_name), source=loc)
if isinstance(node, AsyncExpr):
expr = self._build_expr(node.expression) if node.expression else IRNil()
return IRAsync(expr=expr, source=loc)
if isinstance(node, Lambda):
params = [p.name for p in node.params]
if isinstance(node.body, Expression):
body_expr = self._build_expr(node.body)
return IRLambda(params=params, body_expr=body_expr, source=loc)
else:
body = self._build_block(node.body) if node.body else IRBlock()
return IRLambda(params=params, body=body, source=loc)
if isinstance(node, BaseCall):
args = [self._build_expr(a) for a in node.arguments]
parent = self._current_parent_class
if parent:
this = IRThis(class_name=self._current_class or '', source=loc)
return IRCall(callee_name=f'__ct_class_{parent}_construct',
args=[this] + args, is_shell_cmd=False, source=loc)
return IRCall(callee_name='__base__', args=args,
is_shell_cmd=False, source=loc)
if isinstance(node, (ErrorNode, RangePattern)):
return IRNil(source=loc)
self.errors.append(f'{loc}: Unknown expression type: {type(node).__name__}')
return IRNil(source=loc)
def _build_string(self, node: StringLiteral) -> IRString:
loc = node.location or UNKNOWN_LOC
parts = []
for part in node.parts:
if isinstance(part, StringText):
parts.append(IRStringText(text=part.value, type=T_STRING,
source=part.location or loc))
elif isinstance(part, StringInterp):
expr = self._build_expr(part.expr) if part.expr else IRNil()
parts.append(IRStringInterp(expr=expr, type=T_STRING,
source=part.location or loc))
return IRString(parts=parts, type=T_STRING, source=loc)
def _build_identifier(self, node: Identifier) -> IRExpr:
loc = node.location or UNKNOWN_LOC
name = node.name
sym = self._resolve_name(name)
ty = sym.type if sym else T_ANY
return IRIdentifier(symbol=sym, name=name, type=ty, source=loc)
def _build_call(self, node: CallExpr) -> IRExpr:
loc = node.location or UNKNOWN_LOC
args = [self._build_expr(a) for a in node.arguments]
callee = node.callee
# --- Method call: obj.method(args) ---
if isinstance(callee, MemberAccess):
return self._build_method_call(callee, args, loc)
# --- Simple function call: name(args) ---
if isinstance(callee, Identifier):
name = callee.name
sym = self._resolve_name(name)
if sym is not None:
return IRCall(callee=sym, callee_name=name, args=args,
is_shell_cmd=False, source=loc)
# Builtin (print, len, etc.)
if name in BUILTIN_FUNCS:
return IRCall(callee=None, callee_name=name, args=args,
is_shell_cmd=False, source=loc)
# Unresolved → shell command
return IRCall(callee=None, callee_name=name, args=args,
is_shell_cmd=True, source=loc)
# --- Callee is an expression result (callback variable, etc.) ---
callee_expr = self._build_expr(callee) if callee else IRNil()
# Wrap as a generic call
return IRCall(callee=None, callee_name='__callback__', args=[callee_expr] + args,
is_shell_cmd=False, source=loc)
def _build_method_call(self, ma: MemberAccess, args: list, loc) -> IRExpr:
member = ma.member
obj = ma.object
# --- Namespace call: utils.greet(...) or alias.method(...) ---
if isinstance(obj, Identifier):
obj_name = obj.name
# busing alias: mylib.func() → shell call func()
if obj_name in self._busing:
return IRCall(callee=None, callee_name=member,
args=args, is_shell_cmd=True, source=loc)
ns_name = self._using_aliases.get(obj_name, obj_name)
# Stdlib namespace (fs.read, http.get, ...) — but only if not a user variable
user_sym = self._resolve_name(obj_name)
if ns_name in _STDLIB_NS and user_sym is None:
ns_methods = NAMESPACE_METHODS.get(ns_name)
if ns_methods and member not in ns_methods:
avail = ", ".join(sorted(ns_methods))
self.errors.append(
f"Unknown method '{member}' for type '{ns_name}'. Available: {avail}")
recv = IRIdentifier(name=ns_name, type=T_ANY, source=obj.location or loc)
return IRMethodCall(
receiver=recv, method_name=member,
args=args, kind='stdlib', source=loc,
)
# User namespace: emit as a direct CT function call (no dispatch overhead)
ns_scope = self.global_scope._find_namespace(ns_name)
if ns_scope:
sym = ns_scope.symbols.get(member)
if sym:
return IRCall(callee=sym, callee_name=sym.bash_name(),
args=args, is_shell_cmd=False, source=loc)
# Unknown namespace member → shell command
return IRCall(callee=None, callee_name=f'{ns_name}__{member}',
args=args, is_shell_cmd=True, source=loc)
# --- Regular method call: expr.method(args) ---
recv = self._build_expr(obj) if obj else IRNil()
# stdlib methods on primitive types, or known stdlib method names on any type
if recv.type.kind in ('string', 'array', 'dict') or \
(recv.type.kind in ('any',) and member in _STDLIB_METHOD_NAMES):
type_methods = self._TYPE_METHOD_NAMES.get(recv.type.kind)
if type_methods and member not in type_methods:
avail = ", ".join(sorted(type_methods))
self.errors.append(
f"Unknown method '{member}' for type '{recv.type.kind}'. Available: {avail}")
return IRMethodCall(
receiver=recv, method_name=member,
args=args, kind='stdlib', source=loc,
)
return IRMethodCall(
receiver=recv, method_name=member,
args=args, kind='instance', source=loc,
)
def _build_member_access(self, node: MemberAccess) -> IRExpr:
loc = node.location or UNKNOWN_LOC
# env.VAR read
if isinstance(node.object, Identifier) and node.object.name == 'env':
return IRIdentifier(name=f'env.{node.member}', type=T_STRING, source=loc)
recv = self._build_expr(node.object) if node.object else IRNil()
return IRFieldAccess(receiver=recv, field_name=node.member,
type=T_ANY, source=loc)
# ------------------------------------------------------------------
# Scope helpers
# ------------------------------------------------------------------
def _resolve_name(self, name: str) -> Optional[Symbol]:
sym = self._scope.lookup(name)
if sym: return sym
if name in self._using_names:
return self._using_names[name]
for ns_scope in self._using_scopes:
sym = ns_scope.symbols.get(name)
if sym: return sym
return None
def _find_child_scope(self, kind: str, name: str) -> Optional[Scope]:
for child in self._scope.children:
if child.kind == kind and child.name == name:
return child
return None
# ---------------------------------------------------------------------------
# Convenience
# ---------------------------------------------------------------------------
def build(program: Program, scope: Scope, call_graph: CallGraph,
filename: str = '<stdin>') -> tuple[IRProgram, list[str]]:
b = IRBuilder(scope, call_graph, filename)
ir = b.build(program)
return ir, b.errors
compiler/ir/nodes.py 0 → 100644
View file @ 31bfe20c
"""
Intermediate Representation (IR) nodes.
Key properties:
- Every node has a stable `node_id` (monotonic counter) — safe for CSE maps.
- Every node carries `type: ContenTType` — no type queries during codegen.
- Every node carries `source: SourceLocation` — LSP can always find origin.
- Names are resolved: IRCall.callee is a Symbol, not a string.
- Shell commands are explicit: IRCall.is_shell_cmd = True (no implicit fallback).
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import List, Optional, Literal, Union
import itertools
from ..semantics.types import ContenTType, T_ANY, T_VOID, T_STRING
from ..semantics.scope import Symbol
from ..syntax.nodes import SourceLocation, UNKNOWN_LOC
# ---------------------------------------------------------------------------
# Stable node ID counter
# ---------------------------------------------------------------------------
_id_counter = itertools.count(1)
def _next_id() -> int:
return next(_id_counter)
# ---------------------------------------------------------------------------
# Base
# ---------------------------------------------------------------------------
@dataclass
class IRNode:
node_id: int = field(default_factory=_next_id)
type: ContenTType = field(default_factory=lambda: T_ANY)
source: SourceLocation = field(default_factory=lambda: UNKNOWN_LOC)
# Prevent accidental equality by ID (use `is` for identity)
def __eq__(self, other: object) -> bool: return self is other
def __hash__(self) -> int: return self.node_id
# ---------------------------------------------------------------------------
# Expressions
# ---------------------------------------------------------------------------
@dataclass
class IRInt(IRNode):
value: int = 0
@dataclass
class IRFloat(IRNode):
value: float = 0.0
@dataclass
class IRBool(IRNode):
value: bool = False
@dataclass
class IRNil(IRNode):
pass
@dataclass
class IRString(IRNode):
"""String with pre-lowered interpolation segments."""
parts: List['IRStringPart'] = field(default_factory=list)
@property
def is_plain(self) -> bool:
return all(isinstance(p, IRStringText) for p in self.parts)
@property
def plain_value(self) -> str:
return ''.join(p.text for p in self.parts if isinstance(p, IRStringText))
@dataclass
class IRStringText(IRNode):
text: str = ''
@dataclass
class IRStringInterp(IRNode):
expr: Optional['IRExpr'] = None
IRStringPart = Union[IRStringText, IRStringInterp]
@dataclass
class IRArray(IRNode):
elements: List['IRExpr'] = field(default_factory=list)
@dataclass
class IRDict(IRNode):
pairs: List[tuple['IRExpr', 'IRExpr']] = field(default_factory=list)
@dataclass
class IRIdentifier(IRNode):
"""Reference to a resolved Symbol."""
symbol: Optional[Symbol] = None
name: str = '' # fallback when symbol is None (shell cmd names, builtins)
@dataclass
class IRThis(IRNode):
"""Reference to `this` inside a class method."""
class_name: str = ''
@dataclass
class IRFieldAccess(IRNode):
"""Resolved field access: receiver.field_name."""
receiver: 'IRExpr' = field(default_factory=lambda: IRNil())
field_name: str = ''
field_sym: Optional[Symbol] = None
@dataclass
class IRIndexAccess(IRNode):
object: 'IRExpr' = field(default_factory=lambda: IRNil())
index: 'IRExpr' = field(default_factory=lambda: IRNil())
@dataclass
class IRBinaryOp(IRNode):
left: 'IRExpr' = field(default_factory=lambda: IRNil())
operator: str = ''
right: 'IRExpr' = field(default_factory=lambda: IRNil())
@dataclass
class IRUnaryOp(IRNode):
operator: str = ''
operand: 'IRExpr' = field(default_factory=lambda: IRNil())
@dataclass
class IRCall(IRNode):
"""Function/command call with resolved callee."""
callee: Optional[Symbol] = None # None for unknown/shell commands
callee_name: str = '' # original name (for shell cmds + debug)
args: List['IRExpr'] = field(default_factory=list)
is_shell_cmd: bool = False # explicit: no implicit fallback
@dataclass
class IRMethodCall(IRNode):
"""Method call: receiver.method(args) — resolved to specific method."""
receiver: 'IRExpr' = field(default_factory=lambda: IRNil())
method_name: str = ''
method_sym: Optional[Symbol] = None # resolved method symbol (if known)
args: List['IRExpr'] = field(default_factory=list)
# Type of method: 'instance' (CT class), 'stdlib' (string/array/dict), 'shell'
kind: Literal['instance', 'stdlib', 'shell'] = 'instance'
@dataclass
class IRNew(IRNode):
"""new ClassName(args) — construct a class instance."""
class_sym: Optional[Symbol] = None
class_name: str = ''
args: List['IRExpr'] = field(default_factory=list)
@dataclass
class IRAsync(IRNode):
"""async expr — start a background process."""
expr: 'IRExpr' = field(default_factory=lambda: IRNil())
@dataclass
class IRLambda(IRNode):
params: List[str] = field(default_factory=list)
body: Optional['IRStmt'] = None
body_expr: Optional['IRExpr'] = None # single-expression body
IRExpr = Union[
IRInt, IRFloat, IRBool, IRNil, IRString,
IRStringText, IRStringInterp, IRArray, IRDict,
IRIdentifier, IRThis, IRFieldAccess, IRIndexAccess,
IRBinaryOp, IRUnaryOp,
IRCall, IRMethodCall, IRNew, IRAsync, IRLambda,
]
# ---------------------------------------------------------------------------
# Statements
# ---------------------------------------------------------------------------
@dataclass
class IRAssign(IRNode):
"""Variable assignment: target = value."""
target: str = '' # variable name
value: Optional[IRExpr] = None
op: str = '=' # =, +=, -=, *=, /=
is_local: bool = False # declare as local in bash
is_export: bool = False # export (env.VAR = ...)
@dataclass
class IRFieldAssign(IRNode):
"""Field assignment: obj.field = value."""
receiver: IRExpr = field(default_factory=lambda: IRNil())
field_name: str = ''
value: Optional[IRExpr] = None
op: str = '='
@dataclass
class IRIndexAssign(IRNode):
"""Index assignment: arr[idx] = value."""
object: IRExpr = field(default_factory=lambda: IRNil())
index: IRExpr = field(default_factory=lambda: IRNil())
value: Optional[IRExpr] = None
@dataclass
class IRExprStmt(IRNode):
expr: Optional[IRExpr] = None
@dataclass
class IRReturn(IRNode):
value: Optional[IRExpr] = None
@dataclass
class IRBreak(IRNode):
pass
@dataclass
class IRContinue(IRNode):
pass
@dataclass
class IRBlock(IRNode):
stmts: List['IRStmt'] = field(default_factory=list)
@dataclass
class IRIf(IRNode):
condition: Optional[IRExpr] = None
then_block: Optional[IRBlock] = None
elif_branches: List[tuple[IRExpr, IRBlock]] = field(default_factory=list)
else_block: Optional[IRBlock] = None
@dataclass
class IRWhile(IRNode):
condition: Optional[IRExpr] = None
body: Optional[IRBlock] = None
@dataclass
class IRFor(IRNode):
"""for i in range(...)"""
variable: str = ''
iterable: Optional[IRExpr] = None
body: Optional[IRBlock] = None
@dataclass
class IRForeach(IRNode):
"""foreach item in collection"""
variables: List[str] = field(default_factory=list)
iterable: Optional[IRExpr] = None
body: Optional[IRBlock] = None
# iter_kind: how the iterable is traversed
iter_kind: Literal['array', 'dict', 'process', 'string_split'] = 'array'
@dataclass
class IRWith(IRNode):
variables: List[str] = field(default_factory=list)
resources: List[IRExpr] = field(default_factory=list)
body: Optional[IRBlock] = None
@dataclass
class IRTry(IRNode):
try_block: Optional[IRBlock] = None
except_clauses: List[tuple[Optional[str], Optional[str], IRBlock]] = field(default_factory=list)
finally_block: Optional[IRBlock] = None
@dataclass
class IRThrow(IRNode):
expr: Optional[IRExpr] = None
@dataclass
class IRDefer(IRNode):
expr: Optional[IRExpr] = None
@dataclass
class IRAwait(IRNode):
expr: Optional[IRExpr] = None
@dataclass
class IROnSignal(IRNode):
signal: str = ''
body: Optional[IRBlock] = None
@dataclass
class IRWhenBranch(IRNode):
patterns: List[IRExpr] = field(default_factory=list)
is_else: bool = False
body: Optional[IRBlock] = None
# range_start/end for range patterns (1..10)
ranges: List[tuple[IRExpr, IRExpr]] = field(default_factory=list)
@dataclass
class IRWhen(IRNode):
value: Optional[IRExpr] = None
branches: List[IRWhenBranch] = field(default_factory=list)
IRStmt = Union[
IRAssign, IRFieldAssign, IRIndexAssign,
IRExprStmt, IRReturn, IRBreak, IRContinue, IRBlock,
IRIf, IRWhile, IRFor, IRForeach, IRWith,
IRTry, IRThrow, IRDefer, IRAwait, IROnSignal, IRWhen,
]
# ---------------------------------------------------------------------------
# Top-level declarations
# ---------------------------------------------------------------------------
@dataclass
class IRDecorator(IRNode):
"""Applied decorator with resolved args."""
name: str = ''
obj_name: Optional[str] = None # @obj.method → obj_name='obj'
args: List[tuple[Optional[str], IRExpr]] = field(default_factory=list)
@dataclass
class IRParam(IRNode):
name: str = ''
type: ContenTType = field(default_factory=lambda: T_ANY)
default: Optional[IRExpr] = None
is_variadic: bool = False
@dataclass
class IRFunction(IRNode):
"""A resolved function or class method."""
symbol: Optional[Symbol] = None
name: str = ''
params: List[IRParam] = field(default_factory=list)
body: Optional[IRBlock] = None
decorators: List[IRDecorator] = field(default_factory=list)
is_awk: bool = False
is_method: bool = False
class_name: str = '' # set for methods
is_test: bool = False
@dataclass
class IRClassField(IRNode):
name: str = ''
type: ContenTType = field(default_factory=lambda: T_ANY)
default: Optional[IRExpr] = None
field_kind: Literal['scalar', 'array', 'dict', 'object'] = 'scalar'
@dataclass
class IRClass(IRNode):
symbol: Optional[Symbol] = None
name: str = ''
parent: Optional[str] = None
fields: List[IRClassField] = field(default_factory=list)
constructor: Optional[IRFunction] = None
methods: List[IRFunction] = field(default_factory=list)
@dataclass
class IRBusingImport(IRNode):
name: Optional[str] = None
path: str = ''
@dataclass
class IRUserDecoratorCall(IRNode):
"""User decorator registration call emitted as a top-level statement.
E.g. @router.route('/home') on func home_handler becomes:
__ct_call_method "$router" "route" "/home" "home_handler"
Emitted into top_stmts at the position of the function declaration.
"""
obj_name: str = '' # variable holding the decorator object (e.g. 'router')
method_name: str = '' # decorator method (e.g. 'route')
args: List[IRExpr] = field(default_factory=list) # decorator args (e.g. '/home')
fn_bash_name: str = '' # bash function name to register (e.g. 'home_handler')
@dataclass
class IRProgram(IRNode):
"""Root of the IR tree."""
functions: List[IRFunction] = field(default_factory=list)
classes: List[IRClass] = field(default_factory=list)
top_stmts: List[IRStmt] = field(default_factory=list) # module-level code
busing: List[IRBusingImport] = field(default_factory=list)
filename: str = '<stdin>'
skipped_tests: List[str] = field(default_factory=list) # @test fn names removed by DCE
compiler/lexer/lexer.py 0 → 100644
View file @ 31bfe20c
"""
ContenT lexer — new implementation.
Key differences from bootstrap/lexer.py:
1. String interpolation uses RawText/RawInterp parts instead of \x00LBRACE\x00 hack.
2. Implicit line continuation:
- Inside open parentheses: NEWLINE tokens are suppressed.
- Trailing dot: a line ending with '.' (possibly followed by a comment)
suppresses the following NEWLINE so method chains can span lines.
3. Comments preserved as .trivia on the next token (for LSP).
"""
from __future__ import annotations
from typing import List, Optional
from .tokens import (
Token, TokenType, KEYWORDS,
StringPart, RawText, RawInterp,
)
class LexError(Exception):
def __init__(self, message: str, line: int, col: int, filename: str):
self.message = message
self.line = line
self.col = col
self.filename = filename
super().__init__(f'{filename}:{line}:{col}: {message}')
class Lexer:
def __init__(self, source: str, filename: str = '<stdin>'):
self.source = source
self.filename = filename
self.pos = 0
self.line = 1
self.column = 1
# Implicit line continuation: suppress NEWLINE when paren_depth > 0
self.paren_depth: int = 0
# Accumulated trivia (comment text) to attach to next real token
self._pending_trivia: str = ''
self.errors: list[LexError] = []
# ------------------------------------------------------------------
# Low-level source navigation
# ------------------------------------------------------------------
def _ch(self, offset: int = 0) -> Optional[str]:
p = self.pos + offset
return self.source[p] if p < len(self.source) else None
def _advance(self) -> Optional[str]:
ch = self._ch()
if ch is None:
return None
self.pos += 1
if ch == '\n':
self.line += 1
self.column = 1
else:
self.column += 1
return ch
def _skip_spaces(self):
while self._ch() in (' ', '\t', '\r'):
self._advance()
def _loc(self) -> tuple[int, int]:
return self.line, self.column
# ------------------------------------------------------------------
# Error handling (non-fatal — produce error token and continue)
# ------------------------------------------------------------------
def _error(self, msg: str) -> None:
err = LexError(msg, self.line, self.column, self.filename)
self.errors.append(err)
# ------------------------------------------------------------------
# Token factories
# ------------------------------------------------------------------
def _tok(self, tt: TokenType, value=None, line: int = 0, col: int = 0) -> Token:
t = Token(
type=tt, value=value,
line=line or self.line, column=col or self.column,
trivia=self._pending_trivia,
)
self._pending_trivia = ''
return t
# ------------------------------------------------------------------
# Readers
# ------------------------------------------------------------------
def _read_comment(self) -> str:
"""Read # ... until end of line, return content (without #)."""
self._advance() # skip '#'
parts = []
while self._ch() not in (None, '\n'):
parts.append(self._advance())
return '#' + ''.join(parts)
def _read_string(self, quote: str, line: int, col: int) -> Token:
"""
Read a quoted string, handling escape sequences and {interpolation}.
Returns a STRING token whose value is List[StringPart].
"""
self._advance() # consume opening quote
parts: list[StringPart] = []
text_buf: list[str] = []
def flush_text():
if text_buf:
parts.append(RawText(''.join(text_buf)))
text_buf.clear()
while True:
ch = self._ch()
if ch is None:
self._error('Unterminated string literal')
break
if ch == quote:
self._advance()
break
if ch == '\\':
self._advance()
esc = self._ch()
if esc is None:
self._error('Unterminated escape sequence')
break
escape_map = {
'n': '\n', 't': '\t', 'r': '\r',
'\\': '\\', '"': '"', "'": "'",
'{': '{', '}': '}', '$': '$',
}
text_buf.append(escape_map.get(esc, esc))
self._advance()
continue
if ch == '{':
# Begin interpolation
flush_text()
interp_line, interp_col = self.line, self.column
self._advance() # consume '{'
# Collect raw source of the interpolated expression,
# tracking brace depth so nested {} (e.g. in dict literals) work.
depth = 1
expr_chars: list[str] = []
while depth > 0:
ec = self._ch()
if ec is None:
self._error('Unterminated string interpolation')
break
if ec == '{':
depth += 1
elif ec == '}':
depth -= 1
if depth == 0:
self._advance() # consume closing '}'
break
expr_chars.append(self._advance())
parts.append(RawInterp(
source=''.join(expr_chars),
line=interp_line,
col=interp_col,
))
continue
text_buf.append(self._advance())
flush_text()
return self._tok(TokenType.STRING, parts, line, col)
def _read_number(self, line: int, col: int) -> Token:
digits: list[str] = []
is_float = False
while True:
ch = self._ch()
if ch is None:
break
if ch.isdigit():
digits.append(self._advance())
elif ch == '.' and not is_float and self._ch(1) and self._ch(1).isdigit():
is_float = True
digits.append(self._advance())
else:
break
raw = ''.join(digits)
if is_float:
return self._tok(TokenType.FLOAT, float(raw), line, col)
return self._tok(TokenType.INTEGER, int(raw), line, col)
def _read_identifier(self, line: int, col: int) -> Token:
"""
Read identifier, including hyphenated names like dbus-monitor.
Hyphen is allowed if followed by a letter or underscore.
"""
chars: list[str] = []
while True:
ch = self._ch()
if ch is None:
break
if ch.isalnum() or ch == '_':
chars.append(self._advance())
elif (ch == '-'
and self._ch(1) is not None
and (self._ch(1).isalpha() or self._ch(1) == '_')):
chars.append(self._advance())
else:
break
name = ''.join(chars)
tt = KEYWORDS.get(name, TokenType.IDENTIFIER)
return self._tok(tt, name, line, col)
# ------------------------------------------------------------------
# Main tokenise loop
# ------------------------------------------------------------------
def tokenize(self) -> List[Token]:
tokens: list[Token] = []
SINGLE: dict[str, TokenType] = {
'*': TokenType.STAR, '%': TokenType.PERCENT,
'<': TokenType.LT, '>': TokenType.GT,
'!': TokenType.NOT, ':': TokenType.COLON,
',': TokenType.COMMA, '@': TokenType.AT,
'[': TokenType.LBRACKET, ']': TokenType.RBRACKET,
}
while True:
self._skip_spaces()
ch = self._ch()
line, col = self._loc()
if ch is None:
tokens.append(self._tok(TokenType.EOF, None, line, col))
break
# --- Comment ---
if ch == '#':
comment = self._read_comment()
self._pending_trivia += comment
continue
# --- Newline ---
if ch == '\n':
self._advance()
# Suppress if inside open parens (Python-style implicit continuation)
if self.paren_depth > 0:
continue
# Suppress if next non-space char on new line is '.' but not '..' or '...'
# This enables leading-dot method chain continuation:
# result = obj.method()
# .next_method() ← leading dot, suppress newline before it
tmp = self.pos
while tmp < len(self.source) and self.source[tmp] in (' ', '\t', '\r'):
tmp += 1
if tmp < len(self.source):
nc = self.source[tmp]
nc2 = self.source[tmp + 1] if tmp + 1 < len(self.source) else ''
if nc == '.' and nc2 not in ('.', ''):
continue # leading-dot continuation
tokens.append(self._tok(TokenType.NEWLINE, None, line, col))
continue
# --- String ---
if ch in ('"', "'"):
tokens.append(self._read_string(ch, line, col))
continue
# --- Number ---
if ch.isdigit():
tokens.append(self._read_number(line, col))
continue
# --- Identifier / keyword ---
if ch.isalpha() or ch == '_':
tok = self._read_identifier(line, col)
tokens.append(tok)
continue
# --- Two-char operators ---
peek = self._ch(1)
if ch == '=' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.EQ, '==', line, col))
continue
if ch == '!' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.NEQ, '!=', line, col))
continue
if ch == '<' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.LTE, '<=', line, col))
continue
if ch == '>' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.GTE, '>=', line, col))
continue
if ch == '&' and peek == '&':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.AND, '&&', line, col))
continue
if ch == '|' and peek == '|':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.OR, '||', line, col))
continue
if ch == '=' and peek == '>':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.ARROW, '=>', line, col))
continue
if ch == '+' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.PLUS_ASSIGN, '+=', line, col))
continue
if ch == '-' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.MINUS_ASSIGN, '-=', line, col))
continue
if ch == '*' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.STAR_ASSIGN, '*=', line, col))
continue
if ch == '/' and peek == '=':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.SLASH_ASSIGN, '/=', line, col))
continue
# --- Dot variants (order: ... > .. > .) ---
if ch == '.':
if self._ch(1) == '.' and self._ch(2) == '.':
self._advance(); self._advance(); self._advance()
tokens.append(self._tok(TokenType.DOTDOTDOT, '...', line, col))
self._dot_at_eol = False
elif self._ch(1) == '.':
self._advance(); self._advance()
tokens.append(self._tok(TokenType.DOTDOT, '..', line, col))
self._dot_at_eol = False
else:
self._advance()
tokens.append(self._tok(TokenType.DOT, '.', line, col))
# Mark trailing dot for continuation detection.
# We set it; if another non-whitespace token follows on
# the same line it gets reset to False by that token.
self._dot_at_eol = True
continue
# --- Parens (track depth for implicit continuation) ---
if ch == '(':
self._advance()
self.paren_depth += 1
tokens.append(self._tok(TokenType.LPAREN, '(', line, col))
continue
if ch == ')':
self._advance()
if self.paren_depth > 0:
self.paren_depth -= 1
tokens.append(self._tok(TokenType.RPAREN, ')', line, col))
continue
# --- Brace/bracket ---
if ch == '{':
self._advance()
tokens.append(self._tok(TokenType.LBRACE, '{', line, col))
continue
if ch == '}':
self._advance()
tokens.append(self._tok(TokenType.RBRACE, '}', line, col))
continue
# --- Pipe / OR ---
if ch == '|':
self._advance()
tokens.append(self._tok(TokenType.PIPE, '|', line, col))
continue
# --- Plus / Minus (not compound-assign, handled above) ---
if ch == '+':
self._advance()
tokens.append(self._tok(TokenType.PLUS, '+', line, col))
continue
if ch == '-':
self._advance()
tokens.append(self._tok(TokenType.MINUS, '-', line, col))
continue
# --- Slash ---
if ch == '/':
self._advance()
tokens.append(self._tok(TokenType.SLASH, '/', line, col))
continue
# --- Assign (= not ==, =>; handled above) ---
if ch == '=':
self._advance()
tokens.append(self._tok(TokenType.ASSIGN, '=', line, col))
continue
# --- Single-char tokens ---
if ch in SINGLE:
self._advance()
tokens.append(self._tok(SINGLE[ch], ch, line, col))
continue
# --- Semicolon: treated like newline ---
if ch == ';':
self._advance()
if self.paren_depth == 0:
tokens.append(self._tok(TokenType.NEWLINE, None, line, col))
continue
# --- Unknown ---
self._error(f"Unexpected character: {ch!r}")
self._advance()
return tokens
def tokenize(source: str, filename: str = '<stdin>') -> tuple[list[Token], list[LexError]]:
"""Convenience wrapper: returns (tokens, errors)."""
lex = Lexer(source, filename)
tokens = lex.tokenize()
return tokens, lex.errors
compiler/lexer/tokens.py 0 → 100644
View file @ 31bfe20c
from enum import Enum, auto
from dataclasses import dataclass, field
from typing import Any, List, Optional, Tuple
class TokenType(Enum):
# Literals
INTEGER = auto()
FLOAT = auto()
STRING = auto() # value is List[StringPart] (see below)
TRUE = auto()
FALSE = auto()
NIL = auto()
IDENTIFIER = auto()
# Keywords
FUNC = auto()
CLASS = auto()
CONSTRUCT = auto()
THIS = auto()
BASE = auto()
RETURN = auto()
IF = auto()
ELSE = auto()
FOREACH = auto()
FOR = auto()
IN = auto()
WHILE = auto()
BREAK = auto()
CONTINUE = auto()
TRY = auto()
EXCEPT = auto()
FINALLY = auto()
THROW = auto()
DEFER = auto()
RANGE = auto()
WHEN = auto()
WITH = auto()
NEW = auto()
ASYNC = auto()
AWAIT = auto()
ON = auto()
NAMESPACE = auto()
USING = auto()
BUSING = auto()
# Operators
PLUS = auto()
MINUS = auto()
STAR = auto()
SLASH = auto()
PERCENT = auto()
ASSIGN = auto()
EQ = auto()
NEQ = auto()
LT = auto()
GT = auto()
LTE = auto()
GTE = auto()
AND = auto()
OR = auto()
PIPE = auto()
NOT = auto()
ARROW = auto()
PLUS_ASSIGN = auto()
MINUS_ASSIGN = auto()
STAR_ASSIGN = auto()
SLASH_ASSIGN = auto()
DOT = auto()
DOTDOT = auto()
DOTDOTDOT = auto()
COLON = auto()
# Delimiters
LPAREN = auto()
RPAREN = auto()
LBRACE = auto()
RBRACE = auto()
LBRACKET = auto()
RBRACKET = auto()
COMMA = auto()
# Special
NEWLINE = auto() # significant (statement separator); suppressed inside ()
AT = auto()
COMMENT = auto() # stored as trivia, not emitted to main stream
EOF = auto()
KEYWORDS: dict[str, TokenType] = {
'func': TokenType.FUNC,
'class': TokenType.CLASS,
'construct': TokenType.CONSTRUCT,
'this': TokenType.THIS,
'base': TokenType.BASE,
'return': TokenType.RETURN,
'if': TokenType.IF,
'else': TokenType.ELSE,
'foreach': TokenType.FOREACH,
'for': TokenType.FOR,
'in': TokenType.IN,
'while': TokenType.WHILE,
'break': TokenType.BREAK,
'continue': TokenType.CONTINUE,
'try': TokenType.TRY,
'except': TokenType.EXCEPT,
'finally': TokenType.FINALLY,
'throw': TokenType.THROW,
'defer': TokenType.DEFER,
'range': TokenType.RANGE,
'when': TokenType.WHEN,
'with': TokenType.WITH,
'new': TokenType.NEW,
'async': TokenType.ASYNC,
'await': TokenType.AWAIT,
'on': TokenType.ON,
'namespace': TokenType.NAMESPACE,
'using': TokenType.USING,
'busing': TokenType.BUSING,
'true': TokenType.TRUE,
'false': TokenType.FALSE,
'nil': TokenType.NIL,
}
# ---------------------------------------------------------------------------
# String interpolation parts
# StringPart is stored as the token value for STRING tokens.
# ---------------------------------------------------------------------------
@dataclass
class RawText:
"""Plain text segment of a string."""
text: str
@dataclass
class RawInterp:
"""
Interpolated expression segment: {expr}.
`source` is the raw source text of the expression inside {}.
The parser will re-lex + re-parse it into a proper expression node.
`line` and `col` are the position of the opening { in the source file.
"""
source: str
line: int
col: int
StringPart = RawText | RawInterp
# ---------------------------------------------------------------------------
# Token
# ---------------------------------------------------------------------------
@dataclass
class Token:
type: TokenType
value: Any # for STRING: List[StringPart]; others: raw Python value
line: int
column: int
trivia: str = '' # leading comment/whitespace (for LSP)
def __repr__(self) -> str:
return f'Token({self.type.name}, {self.value!r}, {self.line}:{self.column})'
compiler/meson.build 0 → 100644
View file @ 31bfe20c
install_data(
'__init__.py',
'__main__.py',
'cli.py',
'constants.py',
install_dir: pkgdatadir / 'compiler',
)
install_subdir('lexer', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
install_subdir('syntax', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
install_subdir('semantics', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
install_subdir('ir', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
install_subdir('optimizer', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
install_data(
'backend/__init__.py',
install_dir: pkgdatadir / 'compiler' / 'backend',
)
install_subdir('backend/bash', install_dir: pkgdatadir / 'compiler' / 'backend', exclude_directories: ['__pycache__'])
install_subdir('backend/awk', install_dir: pkgdatadir / 'compiler' / 'backend', exclude_directories: ['__pycache__'])
install_subdir('methods', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
install_subdir('symbols', install_dir: pkgdatadir / 'compiler', exclude_directories: ['__pycache__'])
compiler/optimizer/cse.py 0 → 100644
View file @ 31bfe20c
"""
Common Subexpression Elimination (CSE).
Key improvement over bootstrap: uses stable node_id, NOT Python id().
Any IR transformation preserves node_id, so CSE mappings are stable.
Scope: within a single function body (basic block CSE).
Targets: method calls and function calls that are side-effect-free.
Strategy:
1. Before if/while conditions: pre-compute expensive sub-expressions into
temp variables (__ct_cse_N).
2. Within a basic block: deduplicate identical calls.
A call is considered side-effect-free (CSE-eligible) if:
- It's an IRMethodCall on a known pure type (string/array methods)
- It's an IRCall to a known pure function (math.*, len)
"""
from __future__ import annotations
from typing import Optional
import itertools
from ..ir.nodes import (
IRProgram, IRFunction, IRClass, IRBlock, IRStmt, IRExpr,
IRMethodCall, IRCall, IRAssign, IRIf, IRWhile, IRExprStmt,
IRIdentifier, IRInt,
)
from ..semantics.types import T_ANY
_cse_counter = itertools.count(1)
def _cse_tmp() -> str:
return f'__ct_cse_{next(_cse_counter)}'
# Methods considered pure (no side effects, safe to hoist)
_PURE_METHODS = frozenset({
'upper', 'lower', 'trim', 'len', 'contains', 'starts', 'ends',
'index', 'charAt', 'substr', 'urlencode',
'join', 'slice', 'get',
'abs', 'min', 'max',
})
# Builtin functions considered pure
_PURE_FUNCS = frozenset({'len', 'range'})
def _is_pure_call(node: IRExpr) -> bool:
if isinstance(node, IRMethodCall):
return node.method_name in _PURE_METHODS
if isinstance(node, IRCall) and not node.is_shell_cmd:
return node.callee_name in _PURE_FUNCS
return False
def run_cse(ir: IRProgram) -> IRProgram:
"""Apply CSE to all functions and class methods."""
for fn in ir.functions:
if fn.body:
fn.body = _cse_block(fn.body)
for cl in ir.classes:
if cl.constructor and cl.constructor.body:
cl.constructor.body = _cse_block(cl.constructor.body)
for m in cl.methods:
if m.body:
m.body = _cse_block(m.body)
return ir
def _cse_block(block: IRBlock) -> IRBlock:
"""
Apply CSE within a block:
- Before each if/while: hoist condition sub-expressions to temps.
- Track already-computed expressions within the block.
"""
new_stmts: list[IRStmt] = []
# Map: (method_name, receiver_node_id) → temp_var_name
# Only valid within the current straight-line block segment
computed: dict[tuple, str] = {}
for stmt in block.stmts:
if isinstance(stmt, (IRIf, IRWhile)):
# Hoist pure calls in condition to temps
hoisted, new_cond = _hoist_condition(stmt.condition, computed)
new_stmts.extend(hoisted)
stmt.condition = new_cond
if isinstance(stmt, IRIf):
if stmt.then_block:
stmt.then_block = _cse_block(stmt.then_block)
if stmt.else_block:
stmt.else_block = _cse_block(stmt.else_block)
stmt.elif_branches = [
(c, _cse_block(b)) for c, b in stmt.elif_branches
]
# After a branch: clear computed (control flow merges)
computed.clear()
elif isinstance(stmt, IRWhile):
if stmt.body:
# Re-emit hoisted computations inside loop body too
stmt.body = _cse_block(stmt.body)
computed.clear() # loop may repeat
new_stmts.append(stmt)
block.stmts = new_stmts
return block
def _hoist_condition(
cond: Optional[IRExpr],
computed: dict[tuple, str],
) -> tuple[list[IRStmt], Optional[IRExpr]]:
"""
Find pure sub-calls in `cond`, assign them to temp vars, return
(list_of_assign_stmts, rewritten_condition).
"""
if cond is None:
return [], cond
hoisted: list[IRStmt] = []
new_cond = _rewrite_expr(cond, hoisted, computed)
return hoisted, new_cond
def _rewrite_expr(
node: IRExpr,
hoisted: list[IRStmt],
computed: dict[tuple, str],
) -> IRExpr:
"""Recursively rewrite pure calls to temp variables."""
if _is_pure_call(node):
key = _expr_key(node)
if key:
if key in computed:
# Already computed: replace with existing temp
tmp = computed[key]
else:
# New computation: hoist it
tmp = _cse_tmp()
computed[key] = tmp
hoisted.append(IRAssign(
target=tmp, value=node,
is_local=True,
source=node.source,
))
return IRIdentifier(name=tmp, type=node.type, source=node.source)
# Recurse into children
_rewrite_children(node, hoisted, computed)
return node
def _rewrite_children(
node: IRExpr,
hoisted: list[IRStmt],
computed: dict[tuple, str],
) -> None:
from dataclasses import fields
for f in fields(node):
val = getattr(node, f.name)
if hasattr(val, '__dataclass_fields__'):
new_val = _rewrite_expr(val, hoisted, computed)
setattr(node, f.name, new_val)
elif isinstance(val, list):
new_list = []
for item in val:
if hasattr(item, '__dataclass_fields__'):
new_list.append(_rewrite_expr(item, hoisted, computed))
else:
new_list.append(item)
setattr(node, f.name, new_list)
def _expr_key(node: IRExpr) -> Optional[tuple]:
"""
Return a hashable key for a pure expression, or None if not hashable.
The key must uniquely identify the computation.
"""
if isinstance(node, IRMethodCall):
recv_key = _receiver_key(node.receiver)
if recv_key is None:
return None
return ('method', recv_key, node.method_name)
if isinstance(node, IRCall):
return ('func', node.callee_name)
return None
def _receiver_key(node: IRExpr) -> Optional[str]:
"""Return a string key for a receiver expression, if stable."""
if isinstance(node, IRIdentifier):
return node.name
if isinstance(node, IRInt):
return str(node.value)
return None
compiler/optimizer/dce.py 0 → 100644
View file @ 31bfe20c
"""
Dead Code Elimination — IR level.
Algorithm: BFS reachability from root symbols (main + @test functions + top_stmts).
Uses the CallGraph built by the Resolver.
Replaces 580 lines of bootstrap/dce.py with ~50 lines.
Additionally: for each reachable class, marks all its methods as reachable
(conservative — we don't have full type-aware dispatch yet; TypeChecker will
improve this in a later pass).
Stdlib category tracking: returns a set[str] of used stdlib categories so
BashBackend knows what stdlib functions to emit.
"""
from __future__ import annotations
from typing import Set
from ..ir.nodes import IRProgram, IRFunction, IRClass, IRCall, IRMethodCall, IRNew
from ..semantics.scope import Symbol, CallGraph
# Mapping from stdlib method names → stdlib category for BashBackend
_METHOD_CATEGORIES: dict[str, str] = {
# string
'upper': 'string', 'lower': 'string', 'trim': 'string', 'len': 'string',
'contains': 'string', 'starts': 'string', 'ends': 'string',
'replace': 'string', 'split': 'string', 'substr': 'string',
'index': 'string', 'charAt': 'string', 'urlencode': 'string',
# array
'push': 'array', 'pop': 'array', 'shift': 'array', 'join': 'array',
'slice': 'array', 'map': 'array', 'filter': 'array', 'get': 'array',
'set': 'array',
# dict
'has': 'dict', 'del': 'dict', 'keys': 'dict',
# file
'read': 'fs', 'write': 'fs', 'writeln': 'fs', 'readline': 'fs', 'close': 'fs',
# http
'post': 'http', 'put': 'http', 'delete': 'http',
}
_NS_CATEGORIES: dict[str, str] = {
'http': 'http',
'fs': 'fs',
'json': 'json',
'logger': 'logger',
'reflect': 'reflect',
'regex': 'regex',
'math': 'math',
'time': 'time',
'args': 'args',
}
def run_dce(ir: IRProgram, call_graph: CallGraph,
keep_tests: bool = False) -> tuple[IRProgram, set[str]]:
"""
Remove unreachable functions and classes from ir.
Returns (modified_ir, used_stdlib_categories).
"""
# Collect root symbols
roots: set[Symbol] = set()
# main function
for fn in ir.functions:
if fn.name == 'main' and fn.symbol:
roots.add(fn.symbol)
if keep_tests and fn.is_test and fn.symbol:
roots.add(fn.symbol)
# Functions with user decorators are registered by name (e.g. @router.route)
# and called dynamically — DCE cannot trace them via call graph.
if fn.symbol and any(d.obj_name for d in fn.decorators):
roots.add(fn.symbol)
# top-level statements reference everything they call — mark all their callees
# by walking top_stmts and collecting called symbols
top_refs = _collect_refs_from_stmts(ir.top_stmts, ir)
roots.update(top_refs)
# BFS reachability
reachable = call_graph.reachable_from(roots)
# Also add roots themselves (some may have no outgoing edges)
reachable.update(roots)
# For each reachable class: keep all its methods (conservative)
reachable_class_names: set[str] = set()
for sym in list(reachable):
if sym.kind == 'class':
reachable_class_names.add(sym.name)
# Always keep classes that appear in top_stmts (new Foo())
for stmt in ir.top_stmts:
_collect_class_refs(stmt, reachable_class_names)
# Keep parent classes of reachable classes (needed for inherited method aliases)
cls_by_name = {cl.name: cl for cl in ir.classes}
worklist = list(reachable_class_names)
while worklist:
cname = worklist.pop()
cl = cls_by_name.get(cname)
if cl and cl.parent and cl.parent not in reachable_class_names:
reachable_class_names.add(cl.parent)
worklist.append(cl.parent)
# Determine used stdlib categories (before filtering, to detect reflect usage)
used_categories = _collect_stdlib_categories(ir)
# When reflect or json is used, all classes must survive DCE:
# reflect.create("Item") and json.unmarshal(str, ClassName) pass class names
# as string literals or call constructors dynamically — DCE can't trace them.
if 'reflect' in used_categories or 'json' in used_categories:
for cl in ir.classes:
reachable_class_names.add(cl.name)
# Track @test functions being removed for backend comment emission
ir.skipped_tests = [
fn.name for fn in ir.functions
if fn.is_test and not _fn_is_reachable(fn, reachable, reachable_class_names)
]
# Filter functions
ir.functions = [
fn for fn in ir.functions
if _fn_is_reachable(fn, reachable, reachable_class_names)
]
# Filter classes
ir.classes = [
cl for cl in ir.classes
if cl.name in reachable_class_names or cl.symbol in reachable
]
return ir, used_categories
def _fn_is_reachable(fn: IRFunction, reachable: set[Symbol],
reachable_class_names: set[str]) -> bool:
if fn.symbol and fn.symbol in reachable:
return True
if fn.is_method and fn.class_name in reachable_class_names:
return True
if fn.name == 'main':
return True
return False
def _collect_refs_from_stmts(stmts, ir: IRProgram) -> set[Symbol]:
"""Walk top-level statements and collect referenced symbols."""
refs: set[Symbol] = set()
for stmt in stmts:
_walk_ir_node(stmt, refs)
return refs
def _collect_class_refs(node, class_names: set[str]) -> None:
"""Find new ClassName() in an IR node."""
from ..ir.nodes import IRNew, IRBlock, IRIf, IRWhile, IRFor, IRForeach
from ..ir.nodes import IRWith, IRTry, IRExprStmt, IRAssign, IRReturn
if isinstance(node, IRNew):
class_names.add(node.class_name)
# Recurse into containers (simplified — covers most cases)
for attr in ('stmts', 'then_block', 'else_block', 'body', 'try_block'):
child = getattr(node, attr, None)
if child is None: continue
if isinstance(child, list):
for item in child:
_collect_class_refs(item, class_names)
else:
_collect_class_refs(child, class_names)
def _walk_ir_node(node, refs: set[Symbol],
_visited: set[int] | None = None) -> None:
"""Recursively collect Symbol references from an IR node."""
from ..ir.nodes import (IRNode as _IRNode, IRCall, IRIdentifier, IRNew)
from dataclasses import fields as _dc_fields
if not isinstance(node, _IRNode):
return
if _visited is None:
_visited = set()
nid = node.node_id
if nid in _visited:
return
_visited.add(nid)
if isinstance(node, IRCall) and node.callee:
refs.add(node.callee)
elif isinstance(node, IRIdentifier) and node.symbol:
refs.add(node.symbol)
elif isinstance(node, IRNew) and node.class_sym:
refs.add(node.class_sym)
for f in _dc_fields(node):
val = getattr(node, f.name)
if isinstance(val, _IRNode):
_walk_ir_node(val, refs, _visited)
elif isinstance(val, list):
for item in val:
if isinstance(item, _IRNode):
_walk_ir_node(item, refs, _visited)
elif isinstance(item, tuple):
for sub in item:
if isinstance(sub, _IRNode):
_walk_ir_node(sub, refs, _visited)
def _collect_stdlib_categories(ir: IRProgram) -> set[str]:
"""
Walk IR and detect which stdlib categories are used.
Uses node_id to avoid revisiting Symbol/Scope back-references.
"""
cats: set[str] = {'core'}
visited: set[int] = set()
from ..ir.nodes import IRNode as _IRNode
from dataclasses import fields as _dc_fields
def walk(node) -> None:
if not isinstance(node, _IRNode):
return
nid = node.node_id
if nid in visited:
return
visited.add(nid)
from ..ir.nodes import IRCall, IRMethodCall, IRTry, IRThrow, IRNew
if isinstance(node, (IRTry, IRThrow)):
cats.add('exception')
if isinstance(node, IRNew):
cats.add('object')
if isinstance(node, IRCall) and not node.is_shell_cmd:
name = node.callee_name
if name in ('print', 'len', 'range', 'pid', 'random', 'random_range'):
cats.add('core')
if name in ('assert', 'assert_eq'):
cats.add('test')
elif isinstance(node, IRMethodCall):
cats.update(_method_to_categories(node.method_name, node.receiver))
for f in _dc_fields(node):
val = getattr(node, f.name)
if isinstance(val, _IRNode):
walk(val)
elif isinstance(val, list):
for item in val:
if isinstance(item, _IRNode):
walk(item)
elif isinstance(item, tuple):
for sub in item:
if isinstance(sub, _IRNode):
walk(sub)
for fn in ir.functions:
if fn.is_awk:
cats.add('awk')
if fn.body: walk(fn.body)
for cl in ir.classes:
for m in cl.methods:
if m.is_awk:
cats.add('awk')
if m.body: walk(m.body)
if cl.constructor and cl.constructor.body:
walk(cl.constructor.body)
for stmt in ir.top_stmts:
walk(stmt)
# Classes always need object system
if ir.classes:
cats.add('object')
# Busing always needs core
if ir.busing:
cats.add('misc')
return cats
_METHOD_MULTI_CATEGORIES: dict[str, set[str]] = {
'len': {'string', 'array'},
'get': {'array', 'dict'},
'set': {'array', 'dict'},
}
def _method_to_categories(method: str, receiver) -> set[str]:
from ..ir.nodes import IRIdentifier
# Namespace-based categories
if isinstance(receiver, IRIdentifier):
name = receiver.name
if name in _NS_CATEGORIES:
return {_NS_CATEGORIES[name]}
if name.startswith('env.'):
return {'core'}
# Type-based categories (more precise than method name alone)
if hasattr(receiver, 'type') and receiver.type:
kind = receiver.type.kind
if kind == 'array':
return {'array'}
if kind == 'dict':
return {'dict'}
if kind == 'string':
return {'string'}
if kind == 'any':
# Conservative: emit all possible categories for ambiguous methods
multi = _METHOD_MULTI_CATEGORIES.get(method)
if multi:
return multi
# Method-name based fallback
cat = _METHOD_CATEGORIES.get(method)
return {cat} if cat else set()
compiler/optimizer/fold.py 0 → 100644
View file @ 31bfe20c
"""
Constant folding optimizer.
Walks the IR and replaces:
- BinaryOp(IntLit, op, IntLit) → IntLit
- BinaryOp(FloatLit, op, FloatLit) → FloatLit
- BinaryOp(BoolLit, '&&'/'||', BoolLit) → BoolLit
- UnaryOp('!', BoolLit) → BoolLit
- UnaryOp('-', IntLit/FloatLit) → literal
- IfStmt(BoolLit(True)) → then_block only (dead branch elim)
- IfStmt(BoolLit(False)) → else_block only
- StringLiteral with all-text parts → single IRStringText
Transforms are in-place where possible.
"""
from __future__ import annotations
from typing import Optional
from ..ir.nodes import (
IRNode, IRProgram, IRFunction, IRClass,
IRInt, IRFloat, IRBool, IRNil, IRString, IRStringText,
IRBinaryOp, IRUnaryOp, IRIf, IRBlock,
IRExpr, IRStmt,
)
from ..semantics.types import T_INT, T_FLOAT, T_BOOL, T_STRING
def run_fold(ir: IRProgram) -> IRProgram:
"""Apply constant folding to the entire IR. Returns ir (modified in place)."""
for fn in ir.functions:
if fn.body:
fn.body = _fold_block(fn.body)
for cl in ir.classes:
if cl.constructor and cl.constructor.body:
cl.constructor.body = _fold_block(cl.constructor.body)
for m in cl.methods:
if m.body:
m.body = _fold_block(m.body)
ir.top_stmts = [_fold_stmt(s) for s in ir.top_stmts]
return ir
def _fold_block(block: IRBlock) -> IRBlock:
new_stmts = []
for stmt in block.stmts:
folded = _fold_stmt(stmt)
if folded is not None:
new_stmts.append(folded)
block.stmts = new_stmts
return block
def _fold_stmt(node: IRStmt) -> Optional[IRStmt]:
from ..ir.nodes import (
IRAssign, IRFieldAssign, IRIndexAssign, IRExprStmt,
IRReturn, IRWhile, IRFor, IRForeach, IRWith,
IRTry, IRThrow, IRDefer, IRAwait, IROnSignal, IRWhen,
)
if isinstance(node, IRIf):
return _fold_if(node)
if isinstance(node, IRWhile):
cond = _fold_expr(node.condition)
# if condition is always False: eliminate whole loop
if isinstance(cond, IRBool) and not cond.value:
return None
node.condition = cond
if node.body:
node.body = _fold_block(node.body)
return node
if isinstance(node, (IRFor, IRForeach)):
if hasattr(node, 'iterable') and node.iterable:
node.iterable = _fold_expr(node.iterable)
if node.body:
node.body = _fold_block(node.body)
return node
if isinstance(node, IRAssign):
if node.value:
node.value = _fold_expr(node.value)
return node
if isinstance(node, (IRFieldAssign, IRIndexAssign)):
if node.value:
node.value = _fold_expr(node.value)
return node
if isinstance(node, IRExprStmt):
if node.expr:
node.expr = _fold_expr(node.expr)
return node
if isinstance(node, IRReturn):
if node.value:
node.value = _fold_expr(node.value)
return node
if isinstance(node, IRBlock):
return _fold_block(node)
if isinstance(node, IRWith):
node.resources = [_fold_expr(r) for r in node.resources]
if node.body:
node.body = _fold_block(node.body)
return node
if isinstance(node, IRTry):
if node.try_block:
node.try_block = _fold_block(node.try_block)
node.except_clauses = [
(et, ev, _fold_block(b)) for et, ev, b in node.except_clauses
]
if node.finally_block:
node.finally_block = _fold_block(node.finally_block)
return node
if isinstance(node, (IRThrow, IRDefer, IRAwait)):
if hasattr(node, 'expr') and node.expr:
node.expr = _fold_expr(node.expr)
return node
if isinstance(node, IROnSignal):
if node.body:
node.body = _fold_block(node.body)
return node
if isinstance(node, IRWhen):
if node.value:
node.value = _fold_expr(node.value)
for branch in node.branches:
branch.patterns = [_fold_expr(p) for p in branch.patterns]
if branch.body:
branch.body = _fold_block(branch.body)
return node
return node
def _fold_if(node: IRIf) -> Optional[IRStmt]:
cond = _fold_expr(node.condition) if node.condition else node.condition
node.condition = cond
if node.then_block:
node.then_block = _fold_block(node.then_block)
if node.else_block:
node.else_block = _fold_block(node.else_block)
node.elif_branches = [
(_fold_expr(c), _fold_block(b)) for c, b in node.elif_branches
]
# Dead branch elimination
if isinstance(cond, IRBool):
if cond.value:
# Always true → return then_block as a plain block
return node.then_block or IRBlock(source=node.source)
else:
# Always false → return else_block (or nothing)
if node.elif_branches:
# First elif becomes the new condition
new_cond, new_then = node.elif_branches[0]
node.condition = new_cond
node.then_block = new_then
node.elif_branches = node.elif_branches[1:]
return node
return node.else_block # may be None → removed
return node
def _fold_expr(node: IRExpr) -> IRExpr:
if isinstance(node, IRBinaryOp):
return _fold_binary(node)
if isinstance(node, IRUnaryOp):
return _fold_unary(node)
if isinstance(node, IRString):
return _fold_string(node)
# Recurse into compound expressions
_fold_expr_children(node)
return node
def _fold_binary(node: IRBinaryOp) -> IRExpr:
left = _fold_expr(node.left)
right = _fold_expr(node.right)
op = node.operator
# Int + Int
if isinstance(left, IRInt) and isinstance(right, IRInt):
v = _eval_int(left.value, op, right.value)
if v is not None:
return IRInt(value=v, type=T_INT, source=node.source)
# Float arithmetic
if isinstance(left, (IRInt, IRFloat)) and isinstance(right, (IRInt, IRFloat)):
lv = float(left.value)
rv = float(right.value)
v = _eval_float(lv, op, rv)
if v is not None:
return IRFloat(value=v, type=T_FLOAT, source=node.source)
# Bool logic
if isinstance(left, IRBool) and isinstance(right, IRBool):
if op == '&&': return IRBool(value=left.value and right.value,
type=T_BOOL, source=node.source)
if op == '||': return IRBool(value=left.value or right.value,
type=T_BOOL, source=node.source)
# String concat
if isinstance(left, IRString) and left.is_plain and isinstance(right, IRString) and right.is_plain:
if op == '+':
combined = left.plain_value + right.plain_value
return IRString(
parts=[IRStringText(text=combined, type=T_STRING, source=node.source)],
type=T_STRING, source=node.source,
)
node.left = left
node.right = right
return node
def _fold_unary(node: IRUnaryOp) -> IRExpr:
operand = _fold_expr(node.operand)
if node.operator == '!' and isinstance(operand, IRBool):
return IRBool(value=not operand.value, type=T_BOOL, source=node.source)
if node.operator == '-':
if isinstance(operand, IRInt):
return IRInt(value=-operand.value, type=T_INT, source=node.source)
if isinstance(operand, IRFloat):
return IRFloat(value=-operand.value, type=T_FLOAT, source=node.source)
node.operand = operand
return node
def _fold_string(node: IRString) -> IRExpr:
if node.is_plain:
return node # already plain, nothing to fold
# Try to merge adjacent text parts
new_parts = []
for part in node.parts:
if (isinstance(part, IRStringText) and new_parts and
isinstance(new_parts[-1], IRStringText)):
new_parts[-1].text += part.text
else:
new_parts.append(part)
node.parts = new_parts
return node
def _fold_expr_children(node: IRExpr) -> None:
"""Recursively fold children that are IR nodes (not Symbol/Scope/Type/Location)."""
from ..ir.nodes import IRNode as _IRNode
from dataclasses import fields as _dc_fields
for f in _dc_fields(node):
val = getattr(node, f.name)
if isinstance(val, _IRNode):
setattr(node, f.name, _fold_expr(val))
elif isinstance(val, list):
new_list = []
for item in val:
if isinstance(item, _IRNode):
new_list.append(_fold_expr(item))
elif isinstance(item, tuple):
new_list.append(tuple(
_fold_expr(sub) if isinstance(sub, _IRNode) else sub
for sub in item
))
else:
new_list.append(item)
setattr(node, f.name, new_list)
def _eval_int(a: int, op: str, b: int) -> Optional[int]:
try:
if op == '+': return a + b
if op == '-': return a - b
if op == '*': return a * b
if op == '/': return a // b if b != 0 else None
if op == '%': return a % b if b != 0 else None
except Exception:
pass
return None
def _eval_float(a: float, op: str, b: float) -> Optional[float]:
try:
if op == '+': return a + b
if op == '-': return a - b
if op == '*': return a * b
if op == '/': return a / b if b != 0 else None
except Exception:
pass
return None
compiler/semantics/checker.py 0 → 100644
View file @ 31bfe20c
from __future__ import annotations
from dataclasses import dataclass, field
from typing import List
from .types import ContenTType, T_ANY, T_ERROR
from .scope import Scope, Symbol
from ..ir.nodes import (
IRProgram, IRFunction, IRClass, IRBlock, IRStmt, IRExpr,
IRAssign, IRParam,
)
@dataclass
class TypeDiagnostic:
message: str
filename: str = '<stdin>'
line: int = 0
column: int = 0
def __str__(self) -> str:
loc = f"{self.filename}:{self.line}:{self.column}" if self.line else self.filename
return f"{loc}: {self.message}"
def _type_name(t: ContenTType) -> str:
if t.kind == 'array':
return 'array'
if t.kind == 'dict':
return 'dict'
return t.kind
def _compatible(expected: ContenTType, got: ContenTType) -> bool:
if expected.kind == 'any' or got.kind == 'any':
return True
if expected.kind == 'error' or got.kind == 'error':
return True
if expected.kind == got.kind:
return True
if expected.kind == 'float' and got.kind == 'int':
return True
return False
def check_types(ir: IRProgram, root_scope: Scope) -> List[TypeDiagnostic]:
diags: List[TypeDiagnostic] = []
for sym in root_scope.all_symbols():
if sym.kind in ('var', 'field') and sym.type and sym.type.kind != 'any':
if sym.decl is None:
continue
val_type = _infer_value_type(sym)
if val_type and val_type.kind != 'any' and not _compatible(sym.type, val_type):
loc = sym.defined_at
diags.append(TypeDiagnostic(
message=f"Type mismatch: expected '{_type_name(sym.type)}', got '{_type_name(val_type)}'",
filename=loc.filename if loc else '<stdin>',
line=loc.line if loc else 0,
column=loc.column if loc else 0,
))
return diags
def _infer_value_type(sym: Symbol) -> ContenTType | None:
from ..syntax.nodes import (
Assignment, ClassField,
StringLiteral, IntegerLiteral, FloatLiteral, BoolLiteral, NilLiteral,
ArrayLiteral, DictLiteral, Identifier,
)
decl = sym.decl
if isinstance(decl, Assignment) and decl.value:
return _expr_type(decl.value)
if isinstance(decl, ClassField) and decl.default:
return _expr_type(decl.default)
return None
def _expr_type(node) -> ContenTType | None:
from ..syntax.nodes import (
StringLiteral, IntegerLiteral, FloatLiteral, BoolLiteral, NilLiteral,
ArrayLiteral, DictLiteral,
)
from .types import T_STRING, T_INT, T_FLOAT, T_BOOL, array_of, dict_of
if isinstance(node, StringLiteral):
return T_STRING
if isinstance(node, IntegerLiteral):
return T_INT
if isinstance(node, FloatLiteral):
return T_FLOAT
if isinstance(node, BoolLiteral):
return T_BOOL
if isinstance(node, ArrayLiteral):
return array_of(T_ANY)
if isinstance(node, DictLiteral):
return dict_of(T_ANY, T_ANY)
return None
compiler/semantics/resolver.py 0 → 100644
View file @ 31bfe20c
"""
Name Resolver — Фаза 3.
Traverses the SyntaxTree in two passes:
Pass 1 (collect): register all declarations (funcs, classes, namespaces)
in the global scope and namespace scopes.
Pass 2 (resolve): resolve all name usages to Symbol references.
Build the CallGraph.
Result: (ScopeTree, CallGraph, resolved_usages).
The resolved usages are stored directly on Symbol.usages for LSP.
Shell commands (implicit): any call to a name that is not in scope and
not in BUILTIN_FUNCS is marked as is_shell=True. No implicit fallback —
caller receives an explicit flag.
"""
from __future__ import annotations
from typing import Optional, Union
from ..syntax.nodes import (
Program, FunctionDecl, ClassDecl, ConstructorDecl,
NamespaceDecl, UsingStmt, BusingStmt,
Block, ExpressionStmt, Assignment, ReturnStmt,
IfStmt, WhileStmt, ForStmt, ForeachStmt, WithStmt,
TryStmt, ThrowStmt, DeferStmt, AwaitStmt, OnSignalStmt, WhenStmt,
BreakStmt, ContinueStmt,
CallExpr, MemberAccess, IndexAccess, BinaryOp, UnaryOp,
Identifier, ThisExpr, NewExpr, AsyncExpr, Lambda, BaseCall,
ArrayLiteral, DictLiteral, StringLiteral, StringInterp,
IntegerLiteral, FloatLiteral, BoolLiteral, NilLiteral,
Parameter, ClassField, Decorator, ErrorNode,
SyntaxNode, Expression, Statement, Declaration,
)
from .scope import Scope, Symbol, CallGraph
from .types import (
ContenTType, T_ANY, T_VOID, T_STRING, T_INT, T_BOOL, T_FLOAT,
PRIMITIVE_TYPES, array_of, dict_of, class_type, func_type, from_annotation,
)
# Built-in function names that are always available (from stdlib)
BUILTIN_FUNCS = frozenset({
'print', 'exit', 'len', 'range', 'random', 'random_range', 'pid',
'assert', 'assert_eq',
'http', 'fs', 'json', 'logger', 'reflect', 'regex', 'math',
'time', 'args', 'env', 'shell',
})
# Names that are namespace objects (not functions), accessed via dot
STDLIB_NAMESPACES = frozenset({
'http', 'fs', 'json', 'logger', 'reflect', 'regex', 'math',
'time', 'args', 'env', 'shell',
})
class ResolveError:
def __init__(self, message: str, node: SyntaxNode):
self.message = message
self.node = node
loc = node.location
self.loc_str = f'{loc}' if loc else '?'
def __str__(self) -> str:
return f'{self.loc_str}: {self.message}'
class Resolver:
def __init__(self, filename: str = '<stdin>'):
self.filename = filename
self.errors: list[ResolveError] = []
self.call_graph = CallGraph()
# Global scope (root of scope tree)
self.global_scope = Scope(kind='global', name=None)
# Current resolution context
self._scope: Scope = self.global_scope
self._current_func: Optional[Symbol] = None
self._current_class: Optional[Symbol] = None
# Tracking active using-imports: name → namespace scope
self._using_scopes: list[Scope] = []
# Alias → namespace name (using h = handlers)
self._using_aliases: dict[str, str] = {}
# Selective imports: imported_name → Symbol
self._using_names: dict[str, Symbol] = {}
# Bash libs imported via busing: name → path
self._busing: dict[str, str] = {}
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def resolve(self, program: Program) -> tuple[Scope, CallGraph]:
"""Main entry point. Returns (global_scope, call_graph)."""
self._pass1_collect(program)
self._pass2_resolve(program)
return self.global_scope, self.call_graph
# ------------------------------------------------------------------
# Pass 1: collect all declarations into scopes
# ------------------------------------------------------------------
def _pass1_collect(self, program: Program) -> None:
for stmt in program.statements:
self._collect_top(stmt)
def _collect_top(self, node: SyntaxNode) -> None:
if isinstance(node, FunctionDecl):
self._collect_func(node, self._scope)
elif isinstance(node, ClassDecl):
self._collect_class(node, self._scope)
elif isinstance(node, NamespaceDecl):
self._collect_namespace(node)
elif isinstance(node, UsingStmt):
pass # handled in pass 2
elif isinstance(node, BusingStmt):
if node.name:
self._busing[node.name] = node.path
# Other top-level statements (assignments, etc.) → no declarations
def _collect_func(self, node: FunctionDecl, scope: Scope,
owner_class: Optional[Symbol] = None) -> Symbol:
is_awk = any(d.name == 'awk' for d in node.decorators)
sym = Symbol(
name=node.name,
kind='method' if owner_class else 'func',
type=T_ANY, # resolved in pass 2 / type checker
decl=node,
defined_at=node.location,
is_awk=is_awk,
)
scope.define(sym)
return sym
def _collect_class(self, node: ClassDecl, scope: Scope) -> Symbol:
sym = Symbol(
name=node.name, kind='class',
type=class_type(node.name),
decl=node, defined_at=node.location,
)
scope.define(sym)
# Create class scope and register fields + methods.
# Note: sym.scope is already set to `scope` (parent) by scope.define() — correct.
# cls_scope is a separate child scope for the class's members.
cls_scope = scope.child('class', name=node.name, owner=sym)
for field in node.fields:
f_sym = Symbol(
name=field.name, kind='field',
type=T_ANY, decl=field, defined_at=field.location,
)
cls_scope.define(f_sym)
for method in node.methods:
self._collect_func(method, cls_scope, owner_class=sym)
return sym
def _collect_namespace(self, node: NamespaceDecl) -> None:
# Find or create namespace scope
ns_scope = self.global_scope._find_namespace(node.name)
if ns_scope is None:
ns_sym = Symbol(
name=node.name, kind='namespace',
type=T_ANY, decl=node, defined_at=node.location,
)
self.global_scope.define(ns_sym)
ns_scope = self.global_scope.child('namespace', name=node.name, owner=ns_sym)
for stmt in node.statements:
if isinstance(stmt, FunctionDecl):
self._collect_func(stmt, ns_scope)
elif isinstance(stmt, ClassDecl):
self._collect_class(stmt, ns_scope)
# ------------------------------------------------------------------
# Pass 2: resolve usages
# ------------------------------------------------------------------
def _pass2_resolve(self, program: Program) -> None:
for stmt in program.statements:
self._resolve_stmt(stmt)
# ------------------------------------------------------------------
# Statement resolution
# ------------------------------------------------------------------
def _resolve_stmt(self, node: SyntaxNode) -> None:
if isinstance(node, (FunctionDecl,)):
self._resolve_func(node)
elif isinstance(node, ClassDecl):
self._resolve_class(node)
elif isinstance(node, NamespaceDecl):
self._resolve_namespace(node)
elif isinstance(node, UsingStmt):
self._apply_using(node)
elif isinstance(node, BusingStmt):
pass # already handled in pass 1
elif isinstance(node, Block):
for s in node.statements:
self._resolve_stmt(s)
elif isinstance(node, ExpressionStmt):
if node.expression:
self._resolve_expr(node.expression)
elif isinstance(node, Assignment):
if node.value:
self._resolve_expr(node.value)
# Define variable in current scope if simple identifier
if isinstance(node.target, Identifier):
existing = self._scope.lookup(node.target.name, local_only=True)
if not existing:
ann_type = self._annotation_to_type(node.type_annotation)
if ann_type is None and node.value:
if isinstance(node.value, ArrayLiteral):
ann_type = array_of(T_ANY)
elif isinstance(node.value, DictLiteral):
ann_type = dict_of(T_ANY, T_ANY)
var_sym = Symbol(
name=node.target.name,
kind='var',
type=ann_type or T_ANY,
decl=node,
defined_at=node.location,
)
self._scope.define(var_sym)
elif node.target:
self._resolve_expr(node.target)
elif isinstance(node, ReturnStmt):
if node.value:
self._resolve_expr(node.value)
elif isinstance(node, IfStmt):
if node.condition: self._resolve_expr(node.condition)
if node.then_branch: self._resolve_stmt(node.then_branch)
for cond, blk in (node.elif_branches or []):
self._resolve_expr(cond)
self._resolve_stmt(blk)
if node.else_branch: self._resolve_stmt(node.else_branch)
elif isinstance(node, WhileStmt):
if node.condition: self._resolve_expr(node.condition)
if node.body: self._resolve_stmt(node.body)
elif isinstance(node, (ForStmt, ForeachStmt)):
if node.iterable: self._resolve_expr(node.iterable)
# Register loop variable(s)
if isinstance(node, ForStmt):
self._define_var(node.variable, node)
else:
for v in node.variables:
self._define_var(v, node)
if node.body: self._resolve_stmt(node.body)
elif isinstance(node, WithStmt):
for res in node.resources:
self._resolve_expr(res)
for v in node.variables:
self._define_var(v, node)
if node.body: self._resolve_stmt(node.body)
elif isinstance(node, TryStmt):
if node.try_block: self._resolve_stmt(node.try_block)
for exc_type, exc_var, blk in (node.except_clauses or []):
if exc_var: self._define_var(exc_var, node)
self._resolve_stmt(blk)
if node.finally_block: self._resolve_stmt(node.finally_block)
elif isinstance(node, ThrowStmt):
if node.expression: self._resolve_expr(node.expression)
elif isinstance(node, DeferStmt):
if node.expression: self._resolve_expr(node.expression)
elif isinstance(node, AwaitStmt):
if node.expression: self._resolve_expr(node.expression)
elif isinstance(node, OnSignalStmt):
if node.body: self._resolve_stmt(node.body)
elif isinstance(node, WhenStmt):
if node.value: self._resolve_expr(node.value)
for branch in (node.branches or []):
for pat in branch.patterns:
self._resolve_expr(pat)
if branch.body: self._resolve_stmt(branch.body)
elif isinstance(node, (BreakStmt, ContinueStmt, ErrorNode)):
pass
def _resolve_func(self, node: FunctionDecl) -> None:
# Look up already-collected symbol
func_sym = self._scope.lookup(node.name)
if func_sym is None:
# Shouldn't happen if pass1 ran, but recover gracefully
func_sym = Symbol(name=node.name, kind='func', type=T_ANY,
decl=node, defined_at=node.location)
self._scope.define(func_sym)
saved_func = self._current_func
self._current_func = func_sym
func_scope = self._scope.child('function', name=node.name, owner=func_sym)
saved_scope = self._scope
self._scope = func_scope
# Register parameters
for param in node.params:
p_type = self._annotation_to_type(param.type_annotation)
p_sym = Symbol(name=param.name, kind='param', type=p_type or T_ANY,
decl=param, defined_at=param.location)
func_scope.define(p_sym)
if param.default:
self._resolve_expr(param.default)
if node.body:
self._resolve_stmt(node.body)
self._scope = saved_scope
self._current_func = saved_func
def _resolve_class(self, node: ClassDecl) -> None:
cls_sym = self._scope.lookup(node.name)
if cls_sym is None:
return
saved_class = self._current_class
self._current_class = cls_sym
# Resolve inside class: use the class scope (child of current scope)
cls_scope = next(
(s for s in self._scope.children if s.kind == 'class' and s.name == node.name),
None,
)
if cls_scope is None:
self._current_class = saved_class
return
saved_scope = self._scope
self._scope = cls_scope
# Resolve field defaults
for field in node.fields:
if field.default:
self._resolve_expr(field.default)
# Resolve constructor
if node.constructor:
self._resolve_constructor(node.constructor, cls_scope)
# Resolve methods
for method in node.methods:
saved_func = self._current_func
self._current_func = cls_scope.lookup(method.name)
self._resolve_func(method)
self._current_func = saved_func
self._scope = saved_scope
self._current_class = saved_class
def _resolve_constructor(self, node: ConstructorDecl, cls_scope: Scope) -> None:
saved_scope = self._scope
ctor_scope = cls_scope.child('function', name='construct')
self._scope = ctor_scope
for param in node.params:
p_type = self._annotation_to_type(param.type_annotation)
p_sym = Symbol(name=param.name, kind='param', type=p_type or T_ANY,
decl=param, defined_at=param.location)
ctor_scope.define(p_sym)
if node.body:
self._resolve_stmt(node.body)
self._scope = saved_scope
def _resolve_namespace(self, node: NamespaceDecl) -> None:
ns_scope = self.global_scope._find_namespace(node.name)
if ns_scope is None:
return
saved_scope = self._scope
self._scope = ns_scope
for stmt in node.statements:
self._resolve_stmt(stmt)
self._scope = saved_scope
def _apply_using(self, node: UsingStmt) -> None:
ns_scope = self.global_scope._find_namespace(node.namespace)
if ns_scope is None:
# Might be a stdlib namespace — we still track the alias
pass
if node.alias:
self._using_aliases[node.alias] = node.namespace
if node.names and ns_scope:
for name in node.names:
sym = ns_scope.symbols.get(name)
if sym:
self._using_names[name] = sym
if ns_scope and not node.names:
self._using_scopes.append(ns_scope)
# ------------------------------------------------------------------
# Expression resolution
# ------------------------------------------------------------------
def _resolve_expr(self, node: Expression) -> Optional[Symbol]:
"""Resolve an expression; return the Symbol it refers to (if any)."""
if isinstance(node, Identifier):
return self._resolve_identifier(node)
elif isinstance(node, CallExpr):
return self._resolve_call(node)
elif isinstance(node, MemberAccess):
return self._resolve_member_access(node)
elif isinstance(node, IndexAccess):
if node.object: self._resolve_expr(node.object)
if node.index: self._resolve_expr(node.index)
elif isinstance(node, BinaryOp):
if node.left: self._resolve_expr(node.left)
if node.right: self._resolve_expr(node.right)
elif isinstance(node, UnaryOp):
if node.operand: self._resolve_expr(node.operand)
elif isinstance(node, ArrayLiteral):
for e in node.elements:
self._resolve_expr(e)
elif isinstance(node, DictLiteral):
for k, v in node.pairs:
self._resolve_expr(k)
self._resolve_expr(v)
elif isinstance(node, StringLiteral):
for part in node.parts:
if isinstance(part, StringInterp) and part.expr:
self._resolve_expr(part.expr)
elif isinstance(node, NewExpr):
sym = self._scope.lookup(node.class_name)
if sym:
sym.usages.append(node.location)
if self._current_func:
self.call_graph.add_call(self._current_func, sym)
for arg in node.arguments:
self._resolve_expr(arg)
elif isinstance(node, AsyncExpr):
if node.expression: self._resolve_expr(node.expression)
elif isinstance(node, Lambda):
saved_scope = self._scope
lam_scope = self._scope.child('function', name='<lambda>')
self._scope = lam_scope
for param in node.params:
p_sym = Symbol(name=param.name, kind='param', type=T_ANY,
decl=param, defined_at=param.location)
lam_scope.define(p_sym)
if isinstance(node.body, Expression):
self._resolve_expr(node.body)
elif node.body:
self._resolve_stmt(node.body)
self._scope = saved_scope
elif isinstance(node, BaseCall):
for arg in node.arguments:
self._resolve_expr(arg)
elif isinstance(node, (IntegerLiteral, FloatLiteral, BoolLiteral,
NilLiteral, ThisExpr, ErrorNode)):
pass
return None
def _resolve_identifier(self, node: Identifier) -> Optional[Symbol]:
name = node.name
# 1. Direct scope lookup (current → parent chain)
sym = self._scope.lookup(name)
if sym:
if node.location: sym.usages.append(node.location)
if self._current_func and sym.kind in ('func', 'method', 'class'):
self.call_graph.add_call(self._current_func, sym)
return sym
# 2. using { name } imports
if name in self._using_names:
sym = self._using_names[name]
if node.location: sym.usages.append(node.location)
return sym
# 3. using-imported namespace scopes
for ns_scope in self._using_scopes:
sym = ns_scope.symbols.get(name)
if sym:
if node.location: sym.usages.append(node.location)
if self._current_func:
self.call_graph.add_call(self._current_func, sym)
return sym
# 4. Builtin functions / stdlib namespaces
if name in BUILTIN_FUNCS:
return None # known, no resolution needed
# 5. Not found → treat as shell command (explicit, not crash)
# (IR Builder will set is_shell_cmd=True)
return None
def _resolve_call(self, node: CallExpr) -> Optional[Symbol]:
# First resolve arguments
for arg in node.arguments:
self._resolve_expr(arg)
# Then resolve callee
if isinstance(node.callee, MemberAccess):
return self._resolve_member_call(node)
elif isinstance(node.callee, Identifier):
return self._resolve_identifier(node.callee)
else:
if node.callee: self._resolve_expr(node.callee)
return None
def _resolve_member_access(self, node: MemberAccess) -> Optional[Symbol]:
if node.object: self._resolve_expr(node.object)
# We could do deep type-aware resolution here, but that's TypeChecker's job.
# Resolver just ensures the object expression is resolved.
return None
def _resolve_member_call(self, node: CallExpr) -> Optional[Symbol]:
"""Resolve obj.method() — track call edges when possible."""
ma = node.callee # MemberAccess
# Check if object is a namespace (or alias)
if isinstance(ma.object, Identifier):
obj_name = ma.object.name
# using alias (using h = handlers → h.respond)
ns_name = self._using_aliases.get(obj_name, obj_name)
# Namespace method call
ns_scope = self.global_scope._find_namespace(ns_name)
if ns_scope:
sym = ns_scope.symbols.get(ma.member)
if sym:
if ma.location: sym.usages.append(ma.location)
if self._current_func:
self.call_graph.add_call(self._current_func, sym)
return sym
# busing call (bash library) — mark as shell
if obj_name in self._busing:
return None # IR builder will handle
if ma.object:
self._resolve_expr(ma.object)
return None
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
def _define_var(self, name: str, decl: SyntaxNode,
typ: ContenTType = T_ANY) -> Symbol:
existing = self._scope.lookup(name, local_only=True)
if existing:
return existing
sym = Symbol(name=name, kind='var', type=typ, decl=decl,
defined_at=decl.location if decl else None)
self._scope.define(sym)
return sym
def _annotation_to_type(self, ann) -> Optional[ContenTType]:
if ann is None:
return None
return from_annotation(
name=ann.name,
is_array=ann.is_array,
elem=self._annotation_to_type(ann.element_type),
key=self._annotation_to_type(ann.key_type),
val=self._annotation_to_type(ann.value_type),
)
def resolve(program: Program, filename: str = '<stdin>') -> tuple[Scope, CallGraph, list[ResolveError]]:
"""Convenience wrapper."""
r = Resolver(filename)
scope, call_graph = r.resolve(program)
return scope, call_graph, r.errors
compiler/semantics/scope.py 0 → 100644
View file @ 31bfe20c
"""
Scope tree and Symbol definitions.
Every named entity in a ContenT program is represented as a Symbol.
Symbols live in Scopes which form a tree mirroring the program structure.
Resolver builds this tree; Codegen reads it.
LSP uses it for hover/go-to-def/find-references.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Dict, List, Optional, Literal, TYPE_CHECKING
from .types import ContenTType, T_ANY
if TYPE_CHECKING:
from ..syntax.nodes import SyntaxNode, SourceLocation, FunctionDecl
SymbolKind = Literal[
'func', # top-level or namespace function
'method', # class method
'class', # class declaration
'var', # variable (local or module-level)
'field', # class field
'param', # function parameter
'namespace', # namespace declaration
'busing', # bash library imported via busing
]
ScopeKind = Literal[
'global',
'namespace',
'class',
'function',
'block',
]
@dataclass
class Symbol:
name: str
kind: SymbolKind
type: ContenTType
decl: Optional['SyntaxNode'] # back-reference for go-to-def
defined_at: Optional['SourceLocation']
scope: Optional['Scope'] = None # owning scope (set after creation)
# Flags
is_awk: bool = False # @awk decorated
is_shell: bool = False # shell command (not CT func)
# LSP: all usage sites
usages: List['SourceLocation'] = field(default_factory=list)
def fully_qualified(self) -> str:
"""Return dot-separated fully qualified name, e.g. 'utils.greet'."""
if self.scope and self.scope.kind == 'namespace' and self.scope.name:
return f'{self.scope.name}.{self.name}'
if self.scope and self.scope.kind == 'class' and self.scope.name:
return f'{self.scope.name}.{self.name}'
return self.name
def bash_name(self) -> str:
"""
Return the bash identifier for this symbol.
Namespace prefixing lives ONLY here — IR and Resolver use fully_qualified().
"""
fq = self.fully_qualified()
# utils.greet → utils__greet
# User.getName → __ct_class_User_getName (handled by BashBackend)
return fq.replace('.', '__')
def __hash__(self) -> int:
return id(self)
def __eq__(self, other: object) -> bool:
return self is other
@dataclass
class Scope:
kind: ScopeKind
name: Optional[str] # name for namespace/class/function scopes
parent: Optional['Scope'] = None
symbols: Dict[str, Symbol] = field(default_factory=dict)
children: List['Scope'] = field(default_factory=list)
# For function scopes: reference to the function symbol
owner: Optional[Symbol] = None
def define(self, sym: Symbol) -> None:
"""Add a symbol to this scope."""
sym.scope = self
self.symbols[sym.name] = sym
def lookup(self, name: str, *, local_only: bool = False) -> Optional[Symbol]:
"""Look up a name in this scope and parent scopes."""
if name in self.symbols:
return self.symbols[name]
if local_only or self.parent is None:
return None
return self.parent.lookup(name)
def lookup_qualified(self, ns: str, name: str) -> Optional[Symbol]:
"""Look up ns.name from any scope."""
root = self._root()
ns_scope = root._find_namespace(ns)
if ns_scope:
return ns_scope.symbols.get(name)
return None
def _root(self) -> 'Scope':
s = self
while s.parent:
s = s.parent
return s
def _find_namespace(self, name: str) -> Optional['Scope']:
if self.kind == 'namespace' and self.name == name:
return self
for child in self.children:
found = child._find_namespace(name)
if found:
return found
return None
def child(self, kind: ScopeKind, name: Optional[str] = None,
owner: Optional[Symbol] = None) -> 'Scope':
"""Create and register a child scope."""
s = Scope(kind=kind, name=name, parent=self, owner=owner)
self.children.append(s)
return s
def all_symbols(self) -> List[Symbol]:
"""Collect all symbols in this scope and descendants."""
result = list(self.symbols.values())
for child in self.children:
result.extend(child.all_symbols())
return result
def __repr__(self) -> str:
return f'Scope({self.kind}, {self.name!r}, {list(self.symbols)})'
@dataclass
class CallGraph:
"""
Directed call graph: Symbol → set of Symbols it calls.
Used for DCE (BFS reachability) and optimization.
"""
edges: Dict[Symbol, set[Symbol]] = field(default_factory=dict)
def add_call(self, caller: Symbol, callee: Symbol) -> None:
self.edges.setdefault(caller, set()).add(callee)
def reachable_from(self, roots: set[Symbol]) -> set[Symbol]:
"""BFS: all symbols reachable from roots."""
visited: set[Symbol] = set()
queue = list(roots)
while queue:
sym = queue.pop()
if sym in visited:
continue
visited.add(sym)
for callee in self.edges.get(sym, ()):
if callee not in visited:
queue.append(callee)
return visited
def callees_of(self, sym: Symbol) -> set[Symbol]:
return self.edges.get(sym, set())
compiler/semantics/types.py 0 → 100644
View file @ 31bfe20c
from __future__ import annotations
from dataclasses import dataclass, field
from typing import List, Optional, Literal
Kind = Literal[
'string', 'int', 'float', 'bool', 'void', 'any',
'array', 'dict', 'class', 'func', 'awk', 'error',
]
@dataclass(frozen=True)
class ContenTType:
kind: Kind
element_type: Optional[ContenTType] = None # array<T>
key_type: Optional[ContenTType] = None # dict<K, V>
value_type: Optional[ContenTType] = None # dict<K, V>
class_name: Optional[str] = None # class instance
param_types: tuple[ContenTType, ...] = () # func params
return_type: Optional[ContenTType] = None # func return
def is_scalar(self) -> bool:
return self.kind in ('string', 'int', 'float', 'bool', 'any')
def is_collection(self) -> bool:
return self.kind in ('array', 'dict')
def is_callable(self) -> bool:
return self.kind in ('func', 'awk')
def is_object(self) -> bool:
return self.kind == 'class'
def __repr__(self) -> str:
if self.kind == 'array':
inner = repr(self.element_type) if self.element_type else 'any'
return f'{inner}[]'
if self.kind == 'dict':
k = repr(self.key_type) if self.key_type else 'any'
v = repr(self.value_type) if self.value_type else 'any'
return f'dict[{k}, {v}]'
if self.kind == 'class':
return self.class_name or 'object'
if self.kind == 'func':
params = ', '.join(repr(p) for p in self.param_types)
ret = repr(self.return_type) if self.return_type else 'void'
return f'({params}) => {ret}'
return self.kind
# Prebuilt singletons for common types
T_STRING = ContenTType('string')
T_INT = ContenTType('int')
T_FLOAT = ContenTType('float')
T_BOOL = ContenTType('bool')
T_VOID = ContenTType('void')
T_ANY = ContenTType('any')
T_ERROR = ContenTType('error') # placeholder for unresolved/error types
PRIMITIVE_TYPES: dict[str, ContenTType] = {
'string': T_STRING,
'int': T_INT,
'float': T_FLOAT,
'bool': T_BOOL,
'void': T_VOID,
'any': T_ANY,
}
def array_of(elem: ContenTType) -> ContenTType:
return ContenTType('array', element_type=elem)
def dict_of(key: ContenTType, value: ContenTType) -> ContenTType:
return ContenTType('dict', key_type=key, value_type=value)
def class_type(name: str) -> ContenTType:
return ContenTType('class', class_name=name)
def func_type(params: list[ContenTType], ret: ContenTType) -> ContenTType:
return ContenTType('func', param_types=tuple(params), return_type=ret)
def from_annotation(name: str, is_array: bool = False,
elem: Optional[ContenTType] = None,
key: Optional[ContenTType] = None,
val: Optional[ContenTType] = None) -> ContenTType:
"""Convert a type annotation string to ContenTType."""
if is_array:
inner = PRIMITIVE_TYPES.get(name, class_type(name))
return array_of(inner)
if name == 'array':
return array_of(elem or T_ANY)
if name == 'dict':
k = key or T_STRING
v = val or T_ANY
return dict_of(k, v)
return PRIMITIVE_TYPES.get(name, class_type(name))
compiler/symbols/__init__.py 0 → 100644
View file @ 31bfe20c
from .table import SymbolTable
from .serialize import save, load
compiler/symbols/serialize.py 0 → 100644
View file @ 31bfe20c
"""
JSON serialization / deserialization for SymbolTable.
Saves the full scope tree, symbols, and call graph to a JSON file
so that an LSP server can load it without re-compiling.
Handles circular references (Symbol ↔ Scope) by assigning stable integer IDs.
"""
from __future__ import annotations
import json
from typing import Any, Dict, List, Optional
from ..semantics.scope import Scope, Symbol, CallGraph
from ..semantics.types import ContenTType, PRIMITIVE_TYPES, array_of, dict_of, class_type, func_type
from ..syntax.nodes import SourceLocation
from .table import SymbolTable
def save(table: SymbolTable, path: str) -> None:
data = _encode_table(table)
with open(path, 'w') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
def load(path: str) -> SymbolTable:
with open(path, 'r') as f:
data = json.load(f)
return _decode_table(data)
def _encode_table(table: SymbolTable) -> dict:
ctx = _EncodeContext()
root_id = ctx.encode_scope(table.root_scope)
cg = ctx.encode_call_graph(table.call_graph)
return {
'version': 1,
'filename': table.filename,
'root_scope': root_id,
'scopes': ctx.scopes,
'symbols': ctx.symbols,
'call_graph': cg,
}
class _EncodeContext:
def __init__(self) -> None:
self.scopes: list[dict] = []
self.symbols: list[dict] = []
self._scope_ids: dict[int, int] = {}
self._sym_ids: dict[int, int] = {}
def _sym_id(self, sym: Symbol) -> int:
oid = id(sym)
if oid not in self._sym_ids:
idx = len(self.symbols)
self._sym_ids[oid] = idx
self.symbols.append(self._encode_symbol(sym))
return self._sym_ids[oid]
def _scope_id(self, scope: Scope) -> int:
oid = id(scope)
if oid not in self._scope_ids:
idx = len(self.scopes)
self._scope_ids[oid] = idx
self.scopes.append(None) # type: ignore[arg-type]
self.scopes[idx] = self._encode_scope(scope)
return self._scope_ids[oid]
def encode_scope(self, scope: Scope) -> int:
return self._scope_id(scope)
def _encode_scope(self, scope: Scope) -> dict:
return {
'kind': scope.kind,
'name': scope.name,
'parent': self._scope_id(scope.parent) if scope.parent else None,
'symbols': {name: self._sym_id(sym) for name, sym in scope.symbols.items()},
'children': [self._scope_id(c) for c in scope.children],
'owner': self._sym_id(scope.owner) if scope.owner else None,
}
def _encode_symbol(self, sym: Symbol) -> dict:
return {
'name': sym.name,
'kind': sym.kind,
'type': _encode_type(sym.type),
'defined_at': _encode_loc(sym.defined_at),
'scope': self._scope_id(sym.scope) if sym.scope else None,
'is_awk': sym.is_awk,
'is_shell': sym.is_shell,
'usages': [_encode_loc(u) for u in sym.usages],
}
def encode_call_graph(self, cg: CallGraph) -> list[list[int]]:
result = []
for caller, callees in cg.edges.items():
caller_id = self._sym_id(caller)
for callee in callees:
callee_id = self._sym_id(callee)
result.append([caller_id, callee_id])
return result
def _encode_type(t: ContenTType) -> dict:
d: dict[str, Any] = {'kind': t.kind}
if t.element_type:
d['element_type'] = _encode_type(t.element_type)
if t.key_type:
d['key_type'] = _encode_type(t.key_type)
if t.value_type:
d['value_type'] = _encode_type(t.value_type)
if t.class_name:
d['class_name'] = t.class_name
if t.param_types:
d['param_types'] = [_encode_type(p) for p in t.param_types]
if t.return_type:
d['return_type'] = _encode_type(t.return_type)
return d
def _encode_loc(loc: Optional[SourceLocation]) -> Optional[dict]:
if loc is None:
return None
return {
'line': loc.line,
'column': loc.column,
'filename': loc.filename,
'end_line': loc.end_line,
'end_column': loc.end_column,
}
def _decode_table(data: dict) -> SymbolTable:
ctx = _DecodeContext(data['scopes'], data['symbols'])
root = ctx.decode_scope(data['root_scope'])
cg = ctx.decode_call_graph(data.get('call_graph', []))
return SymbolTable(
root_scope=root,
call_graph=cg,
filename=data.get('filename', '<stdin>'),
)
class _DecodeContext:
def __init__(self, raw_scopes: list[dict], raw_symbols: list[dict]) -> None:
self._raw_scopes = raw_scopes
self._raw_symbols = raw_symbols
self._scopes: dict[int, Scope] = {}
self._symbols: dict[int, Symbol] = {}
def decode_scope(self, idx: int) -> Scope:
if idx in self._scopes:
return self._scopes[idx]
raw = self._raw_scopes[idx]
scope = Scope(kind=raw['kind'], name=raw.get('name'))
self._scopes[idx] = scope
if raw.get('parent') is not None:
scope.parent = self.decode_scope(raw['parent'])
for name, sym_idx in raw.get('symbols', {}).items():
sym = self.decode_symbol(sym_idx)
sym.scope = scope
scope.symbols[name] = sym
for child_idx in raw.get('children', []):
child = self.decode_scope(child_idx)
child.parent = scope
scope.children.append(child)
if raw.get('owner') is not None:
scope.owner = self.decode_symbol(raw['owner'])
return scope
def decode_symbol(self, idx: int) -> Symbol:
if idx in self._symbols:
return self._symbols[idx]
raw = self._raw_symbols[idx]
sym = Symbol(
name=raw['name'],
kind=raw['kind'],
type=_decode_type(raw['type']),
decl=None,
defined_at=_decode_loc(raw.get('defined_at')),
is_awk=raw.get('is_awk', False),
is_shell=raw.get('is_shell', False),
usages=[_decode_loc(u) for u in raw.get('usages', []) if u],
)
self._symbols[idx] = sym
return sym
def decode_call_graph(self, edges: list[list[int]]) -> CallGraph:
cg = CallGraph()
for caller_idx, callee_idx in edges:
caller = self.decode_symbol(caller_idx)
callee = self.decode_symbol(callee_idx)
cg.add_call(caller, callee)
return cg
def _decode_type(d: dict) -> ContenTType:
kind = d['kind']
if kind in PRIMITIVE_TYPES:
return PRIMITIVE_TYPES[kind]
if kind == 'error':
from ..semantics.types import T_ERROR
return T_ERROR
if kind == 'array':
elem = _decode_type(d['element_type']) if d.get('element_type') else None
return ContenTType('array', element_type=elem)
if kind == 'dict':
k = _decode_type(d['key_type']) if d.get('key_type') else None
v = _decode_type(d['value_type']) if d.get('value_type') else None
return ContenTType('dict', key_type=k, value_type=v)
if kind == 'class':
return class_type(d.get('class_name', ''))
if kind == 'func' or kind == 'awk':
params = tuple(_decode_type(p) for p in d.get('param_types', []))
ret = _decode_type(d['return_type']) if d.get('return_type') else None
return ContenTType(kind, param_types=params, return_type=ret)
return ContenTType(kind)
def _decode_loc(d: Optional[dict]) -> Optional[SourceLocation]:
if d is None:
return None
return SourceLocation(
line=d['line'],
column=d['column'],
filename=d.get('filename', '<stdin>'),
end_line=d.get('end_line', 0),
end_column=d.get('end_column', 0),
)
compiler/symbols/table.py 0 → 100644
View file @ 31bfe20c
"""
SymbolTable — unified query interface over Scope tree for LSP.
Built from Resolver output (Scope + CallGraph), provides:
- lookup_at(loc) → find symbol at a source position
- find_references(sym) → all usage sites
- completions_at(loc) → symbols visible at a position
- all_symbols() → flat list of every defined symbol
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Dict, List, Optional
from ..semantics.scope import Scope, Symbol, CallGraph
from ..semantics.types import ContenTType
from ..syntax.nodes import SourceLocation
@dataclass
class SymbolTable:
root_scope: Scope
call_graph: CallGraph
filename: str = '<stdin>'
_by_loc: Dict[tuple, Symbol] = field(default_factory=dict, repr=False)
_by_name: Dict[str, list[Symbol]] = field(default_factory=dict, repr=False)
def __post_init__(self) -> None:
self._index()
def _index(self) -> None:
for sym in self.root_scope.all_symbols():
if sym.defined_at:
key = (sym.defined_at.filename, sym.defined_at.line, sym.defined_at.column)
self._by_loc[key] = sym
self._by_name.setdefault(sym.name, []).append(sym)
def all_symbols(self) -> List[Symbol]:
return self.root_scope.all_symbols()
def lookup_at(self, loc: SourceLocation) -> Optional[Symbol]:
"""Find the symbol defined or referenced at a source position."""
key = (loc.filename, loc.line, loc.column)
sym = self._by_loc.get(key)
if sym:
return sym
for sym in self.root_scope.all_symbols():
for usage in sym.usages:
if (usage.filename == loc.filename
and usage.line == loc.line
and usage.column <= loc.column
and (usage.end_column or usage.column + len(sym.name)) >= loc.column):
return sym
return None
def lookup_name(self, name: str) -> List[Symbol]:
"""Find all symbols with a given name."""
return self._by_name.get(name, [])
def find_references(self, sym: Symbol) -> List[SourceLocation]:
"""All locations where sym is referenced (including definition)."""
refs: List[SourceLocation] = []
if sym.defined_at:
refs.append(sym.defined_at)
refs.extend(sym.usages)
return refs
def completions_at(self, loc: SourceLocation) -> List[Symbol]:
"""Symbols visible at a given source position (for autocomplete)."""
scope = self._scope_at(loc)
if not scope:
scope = self.root_scope
result: list[Symbol] = []
s: Optional[Scope] = scope
while s:
result.extend(s.symbols.values())
s = s.parent
return result
def callers_of(self, sym: Symbol) -> List[Symbol]:
"""Symbols that call sym."""
callers = []
for caller, callees in self.call_graph.edges.items():
if sym in callees:
callers.append(caller)
return callers
def callees_of(self, sym: Symbol) -> List[Symbol]:
"""Symbols that sym calls."""
return list(self.call_graph.callees_of(sym))
def save(self, path: str) -> None:
from .serialize import save
save(self, path)
@classmethod
def load(cls, path: str) -> 'SymbolTable':
from .serialize import load
return load(path)
def _scope_at(self, loc: SourceLocation) -> Optional[Scope]:
"""Find the innermost scope containing loc (heuristic: by defined_at)."""
best: Optional[Scope] = None
best_line = -1
for sym in self.root_scope.all_symbols():
if (sym.scope and sym.defined_at
and sym.defined_at.filename == loc.filename
and sym.defined_at.line <= loc.line
and sym.defined_at.line > best_line):
if sym.kind in ('func', 'method', 'class', 'namespace'):
for child in (sym.scope.children if sym.scope else []):
if child.owner is sym:
best = child
best_line = sym.defined_at.line
return best
compiler/syntax/nodes.py 0 → 100644
View file @ 31bfe20c
"""
SyntaxTree node definitions for the ContenT compiler.
Key differences from bootstrap/ast_nodes.py:
- SourceLocation tracks both start and end positions (for LSP ranges)
- StringLiteral uses StringPart list instead of \x00LBRACE\x00 hack
- ErrorNode allows parser to continue past syntax errors
- Trivia (comments) preserved on nodes for LSP formatting
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import List, Optional, Union
# ---------------------------------------------------------------------------
# Source location
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class SourceLocation:
line: int
column: int
filename: str = '<stdin>'
end_line: int = 0 # 0 means same as line
end_column: int = 0 # 0 means unknown
def __repr__(self) -> str:
return f'{self.filename}:{self.line}:{self.column}'
def to(self, end: 'SourceLocation') -> 'SourceLocation':
"""Extend this location to cover up to end."""
return SourceLocation(
self.line, self.column, self.filename,
end.end_line or end.line, end.end_column or end.column,
)
UNKNOWN_LOC = SourceLocation(0, 0, '<unknown>')
# ---------------------------------------------------------------------------
# Base
# ---------------------------------------------------------------------------
@dataclass
class SyntaxNode:
location: Optional[SourceLocation] = field(default=None, repr=False)
trivia: List[str] = field(default_factory=list, repr=False)
# trivia = leading comments/whitespace, for LSP formatting
@dataclass
class Expression(SyntaxNode):
pass
@dataclass
class Statement(SyntaxNode):
pass
@dataclass
class Declaration(SyntaxNode):
pass
# ---------------------------------------------------------------------------
# Error recovery
# ---------------------------------------------------------------------------
@dataclass
class ErrorNode(SyntaxNode):
"""Placeholder inserted by parser when it encounters a syntax error.
Allows parsing to continue and report multiple errors."""
message: str = ''
raw_tokens: List[str] = field(default_factory=list)
# ---------------------------------------------------------------------------
# Type annotations (source-level, not resolved types)
# ---------------------------------------------------------------------------
@dataclass
class TypeAnnotation(SyntaxNode):
name: str = ''
is_array: bool = False
element_type: Optional['TypeAnnotation'] = None
key_type: Optional['TypeAnnotation'] = None
value_type: Optional['TypeAnnotation'] = None
param_types: List['TypeAnnotation'] = field(default_factory=list)
return_type: Optional['TypeAnnotation'] = None
# ---------------------------------------------------------------------------
# String interpolation
# ---------------------------------------------------------------------------
@dataclass
class StringText(SyntaxNode):
"""A plain text segment inside a string literal."""
value: str = ''
@dataclass
class StringInterp(SyntaxNode):
"""An interpolated expression {expr} inside a string literal."""
expr: Optional[Expression] = None
StringPart = Union[StringText, StringInterp]
# ---------------------------------------------------------------------------
# Expressions
# ---------------------------------------------------------------------------
@dataclass
class IntegerLiteral(Expression):
value: int = 0
@dataclass
class FloatLiteral(Expression):
value: float = 0.0
@dataclass
class StringLiteral(Expression):
"""
String literal with proper part decomposition.
`parts` replaces the old (value, has_interpolation) pair and the
\x00LBRACE\x00 hack in the lexer.
"""
parts: List[StringPart] = field(default_factory=list)
@property
def has_interpolation(self) -> bool:
return any(isinstance(p, StringInterp) for p in self.parts)
@property
def raw_value(self) -> str:
"""Return the string value if there is no interpolation."""
return ''.join(p.value for p in self.parts if isinstance(p, StringText))
@dataclass
class BoolLiteral(Expression):
value: bool = False
@dataclass
class NilLiteral(Expression):
pass
@dataclass
class Identifier(Expression):
name: str = ''
@dataclass
class ArrayLiteral(Expression):
elements: List[Expression] = field(default_factory=list)
@dataclass
class DictLiteral(Expression):
pairs: List[tuple[Expression, Expression]] = field(default_factory=list)
@dataclass
class BinaryOp(Expression):
left: Optional[Expression] = None
operator: str = ''
right: Optional[Expression] = None
@dataclass
class UnaryOp(Expression):
operator: str = ''
operand: Optional[Expression] = None
@dataclass
class CallExpr(Expression):
callee: Optional[Expression] = None
arguments: List[Expression] = field(default_factory=list)
@dataclass
class MemberAccess(Expression):
object: Optional[Expression] = None
member: str = ''
@dataclass
class IndexAccess(Expression):
object: Optional[Expression] = None
index: Optional[Expression] = None
@dataclass
class Lambda(Expression):
params: List['Parameter'] = field(default_factory=list)
body: Union['Block', Expression, None] = None
return_type: Optional[TypeAnnotation] = None
@dataclass
class ThisExpr(Expression):
pass
@dataclass
class BaseCall(Expression):
arguments: List[Expression] = field(default_factory=list)
@dataclass
class NewExpr(Expression):
class_name: str = ''
arguments: List[Expression] = field(default_factory=list)
@dataclass
class AsyncExpr(Expression):
expression: Optional[Expression] = None
@dataclass
class RangePattern(Expression):
"""Range pattern for when branches: 1..10"""
start: Optional[Expression] = None
end: Optional[Expression] = None
# ---------------------------------------------------------------------------
# Statements
# ---------------------------------------------------------------------------
@dataclass
class Block(Statement):
statements: List[Union[Statement, Declaration]] = field(default_factory=list)
@dataclass
class ExpressionStmt(Statement):
expression: Optional[Expression] = None
@dataclass
class Assignment(Statement):
target: Optional[Expression] = None
type_annotation: Optional[TypeAnnotation] = None
operator: str = '='
value: Optional[Expression] = None
@dataclass
class ReturnStmt(Statement):
value: Optional[Expression] = None
@dataclass
class BreakStmt(Statement):
pass
@dataclass
class ContinueStmt(Statement):
pass
@dataclass
class IfStmt(Statement):
condition: Optional[Expression] = None
then_branch: Optional[Block] = None
elif_branches: List[tuple[Expression, Block]] = field(default_factory=list)
else_branch: Optional[Block] = None
@dataclass
class WhileStmt(Statement):
condition: Optional[Expression] = None
body: Optional[Block] = None
@dataclass
class ForStmt(Statement):
variable: str = ''
iterable: Optional[Expression] = None
body: Optional[Block] = None
@dataclass
class ForeachStmt(Statement):
variables: List[str] = field(default_factory=list)
iterable: Optional[Expression] = None
body: Optional[Block] = None
@dataclass
class WithStmt(Statement):
variables: List[str] = field(default_factory=list)
resources: List[Expression] = field(default_factory=list)
body: Optional[Block] = None
@dataclass
class TryStmt(Statement):
try_block: Optional[Block] = None
except_clauses: List[tuple[Optional[str], Block]] = field(default_factory=list)
finally_block: Optional[Block] = None
@dataclass
class ThrowStmt(Statement):
expression: Optional[Expression] = None
@dataclass
class DeferStmt(Statement):
expression: Optional[Expression] = None
@dataclass
class AwaitStmt(Statement):
expression: Optional[Expression] = None
@dataclass
class OnSignalStmt(Statement):
signal: str = ''
body: Optional[Block] = None
@dataclass
class WhenBranch(SyntaxNode):
patterns: List[Expression] = field(default_factory=list)
is_else: bool = False
body: Optional[Block] = None
@dataclass
class WhenStmt(Statement):
value: Optional[Expression] = None
branches: List[WhenBranch] = field(default_factory=list)
# ---------------------------------------------------------------------------
# Module system
# ---------------------------------------------------------------------------
@dataclass
class NamespaceDecl(Statement):
name: str = ''
statements: List[Union[Statement, Declaration]] = field(default_factory=list)
@dataclass
class UsingStmt(Statement):
namespace: str = ''
alias: Optional[str] = None
names: Optional[List[str]] = None # using ns { name1, name2 }
@dataclass
class BusingStmt(Statement):
name: Optional[str] = None # busing config = "/path"
path: str = ''
# ---------------------------------------------------------------------------
# Declarations
# ---------------------------------------------------------------------------
@dataclass
class Parameter(SyntaxNode):
name: str = ''
type_annotation: Optional[TypeAnnotation] = None
default: Optional[Expression] = None
is_variadic: bool = False
@dataclass
class Decorator(SyntaxNode):
"""
Decorator applied to a function or method.
name = builtin decorator name ('retry', 'log', etc.) OR custom method name
object = object name for custom decorators (@router.route → object='router', name='route')
"""
name: str = ''
arguments: List[tuple[Optional[str], Expression]] = field(default_factory=list)
object: Optional[str] = None
@dataclass
class FunctionDecl(Declaration):
name: str = ''
params: List[Parameter] = field(default_factory=list)
return_type: Optional[TypeAnnotation] = None
body: Optional[Block] = None
decorators: List[Decorator] = field(default_factory=list)
@dataclass
class ClassField(SyntaxNode):
name: str = ''
type_annotation: Optional[TypeAnnotation] = None
default: Optional[Expression] = None
@dataclass
class ConstructorDecl(Declaration):
params: List[Parameter] = field(default_factory=list)
body: Optional[Block] = None
@dataclass
class ClassDecl(Declaration):
name: str = ''
parent: Optional[str] = None
fields: List[ClassField] = field(default_factory=list)
constructor: Optional[ConstructorDecl] = None
methods: List[FunctionDecl] = field(default_factory=list)
# ---------------------------------------------------------------------------
# Program root
# ---------------------------------------------------------------------------
@dataclass
class Program(SyntaxNode):
statements: List[Union[Statement, Declaration]] = field(default_factory=list)
filename: str = '<stdin>'
compiler/syntax/parser.py 0 → 100644
View file @ 31bfe20c
"""
ContenT parser — new implementation.
Key differences from bootstrap/parser.py:
1. Error recovery: instead of crashing, inserts ErrorNode and continues parsing.
2. Namespace merging: multiple `namespace X {}` blocks with the same name are
merged into one NamespaceDecl in parse().
3. STRING tokens carry List[StringPart]; parser re-lexes interpolations.
4. Lambda.params uses Parameter objects (supports type annotations).
5. Multi-line is transparent — handled by lexer (implicit continuation).
"""
from __future__ import annotations
from typing import List, Optional, Union
from ..lexer.tokens import Token, TokenType, RawText, RawInterp
from ..lexer.lexer import Lexer
from .nodes import (
SourceLocation, UNKNOWN_LOC,
Program, Declaration, Statement,
Decorator, FunctionDecl, Parameter,
ClassDecl, ClassField, ConstructorDecl,
Block, ReturnStmt, BreakStmt, ContinueStmt,
IfStmt, WhileStmt, ForStmt, ForeachStmt, WithStmt,
TryStmt, ThrowStmt, DeferStmt, AwaitStmt, OnSignalStmt,
WhenStmt, WhenBranch, RangePattern,
ExpressionStmt, Assignment,
IntegerLiteral, FloatLiteral, StringLiteral, BoolLiteral, NilLiteral,
StringText, StringInterp,
ThisExpr, BaseCall, ArrayLiteral, DictLiteral,
Identifier, BinaryOp, UnaryOp, CallExpr,
MemberAccess, IndexAccess, NewExpr, AsyncExpr, Lambda,
Expression, TypeAnnotation,
NamespaceDecl, UsingStmt, BusingStmt,
ErrorNode,
)
class ParseError(Exception):
"""Internal: raised only for unrecoverable situations."""
# Token types that can start a new top-level declaration (sync points)
_SYNC_TOP = {
TokenType.FUNC, TokenType.CLASS, TokenType.NAMESPACE,
TokenType.USING, TokenType.BUSING, TokenType.AT,
TokenType.EOF,
}
# Token types that can start a statement (sync points inside a block)
_SYNC_STMT = _SYNC_TOP | {
TokenType.RETURN, TokenType.BREAK, TokenType.CONTINUE,
TokenType.IF, TokenType.WHILE, TokenType.FOR, TokenType.FOREACH,
TokenType.WITH, TokenType.TRY, TokenType.THROW, TokenType.DEFER,
TokenType.AWAIT, TokenType.ON, TokenType.WHEN,
TokenType.RBRACE, TokenType.NEWLINE,
}
# Keywords that can appear as identifiers in certain positions
_KEYWORD_AS_IDENT = {TokenType.ON, TokenType.ASYNC, TokenType.AWAIT}
class Parser:
def __init__(self, tokens: List[Token], filename: str = '<stdin>'):
self.tokens = tokens
self.filename = filename
self.pos = 0
self.errors: list[str] = [] # (message, line, col)
# ------------------------------------------------------------------
# Token navigation
# ------------------------------------------------------------------
def _cur(self) -> Token:
return self.tokens[min(self.pos, len(self.tokens) - 1)]
def _peek(self, offset: int = 1) -> Token:
p = self.pos + offset
return self.tokens[min(p, len(self.tokens) - 1)]
def _check(self, *types: TokenType) -> bool:
return self._cur().type in types
def _match(self, *types: TokenType) -> bool:
if self._check(*types):
self._advance()
return True
return False
def _advance(self) -> Token:
tok = self._cur()
if tok.type != TokenType.EOF:
self.pos += 1
return tok
def _expect(self, tt: TokenType, message: str = '') -> Token:
if self._check(tt):
return self._advance()
msg = message or f'Expected {tt.name}, got {self._cur().type.name}'
self._error(msg)
return self._cur() # don't consume — let caller sync
def _expect_name(self, message: str = 'Expected identifier') -> Token:
if self._check(TokenType.IDENTIFIER):
return self._advance()
if self._cur().type in _KEYWORD_AS_IDENT:
tok = self._advance()
if not tok.value:
tok.value = tok.type.name.lower()
return tok
self._error(message)
return self._cur()
def _error(self, message: str, tok: Optional[Token] = None) -> None:
t = tok or self._cur()
self.errors.append(f'{self.filename}:{t.line}:{t.column}: {message}')
def _loc(self, tok: Optional[Token] = None) -> SourceLocation:
t = tok or self._cur()
return SourceLocation(t.line, t.column, self.filename)
def _skip_newlines(self) -> None:
while self._match(TokenType.NEWLINE):
pass
def _sync_to(self, sync_set: set[TokenType]) -> None:
"""Skip tokens until one in sync_set is found (for error recovery)."""
while self._cur().type not in sync_set and self._cur().type != TokenType.EOF:
self._advance()
# ------------------------------------------------------------------
# String helpers
# ------------------------------------------------------------------
def _token_raw_str(self, tok: Token) -> str:
"""Return raw text content of a STRING token (ignoring interpolations)."""
parts = tok.value or []
return ''.join(p.text for p in parts if isinstance(p, RawText))
def _parse_string_literal(self, tok: Token) -> StringLiteral:
"""
Convert a STRING token (with List[StringPart]) into a StringLiteral node,
re-parsing each RawInterp expression using a nested Parser.
"""
loc = self._loc(tok)
node_parts = []
for part in (tok.value or []):
if isinstance(part, RawText):
node_parts.append(StringText(value=part.text, location=loc))
else:
interp_loc = SourceLocation(part.line, part.col, self.filename)
expr = self._parse_interp_expr(part.source, interp_loc)
node_parts.append(StringInterp(expr=expr, location=interp_loc))
return StringLiteral(parts=node_parts, location=loc)
def _parse_interp_expr(self, source: str, loc: SourceLocation) -> Expression:
"""Re-lex and re-parse a string interpolation expression."""
from ..lexer.lexer import tokenize as _lex
lex_tokens, lex_errors = _lex(source, self.filename)
for e in lex_errors:
self.errors.append(str(e))
sub = Parser(lex_tokens, self.filename)
expr = sub.parse_expression()
self.errors.extend(sub.errors)
return expr
# ------------------------------------------------------------------
# Program / top-level
# ------------------------------------------------------------------
def parse(self) -> Program:
"""Parse the full token stream into a Program with merged namespaces."""
stmts: list = []
self._skip_newlines()
while not self._check(TokenType.EOF):
stmt = self._parse_declaration()
if stmt:
stmts.append(stmt)
self._skip_newlines()
# Merge namespace blocks with the same name
ns_map: dict[str, NamespaceDecl] = {}
merged: list = []
for stmt in stmts:
if isinstance(stmt, NamespaceDecl):
if stmt.name in ns_map:
ns_map[stmt.name].statements.extend(stmt.statements)
else:
ns_map[stmt.name] = stmt
merged.append(stmt)
else:
merged.append(stmt)
return Program(statements=merged, filename=self.filename,
location=SourceLocation(1, 1, self.filename))
# ------------------------------------------------------------------
# Declarations
# ------------------------------------------------------------------
def _parse_declaration(self) -> Optional[Union[Declaration, Statement]]:
decorators: list[Decorator] = []
while self._check(TokenType.AT):
decorators.append(self._parse_decorator())
self._skip_newlines()
if self._check(TokenType.FUNC):
return self._parse_function(decorators)
if self._check(TokenType.CLASS):
if decorators:
self._error('Decorators cannot be applied to classes')
return self._parse_class()
if self._check(TokenType.NAMESPACE):
if decorators:
self._error('Decorators cannot be applied to namespaces')
return self._parse_namespace()
if self._check(TokenType.USING):
return self._parse_using()
if self._check(TokenType.BUSING):
return self._parse_busing()
if decorators:
self._error('Decorators can only be applied to functions')
return self._parse_statement()
def _parse_decorator(self) -> Decorator:
loc = self._loc()
self._expect(TokenType.AT)
name = self._expect(TokenType.IDENTIFIER, 'Expected decorator name').value
obj = None
if self._match(TokenType.DOT):
obj = name
name = self._expect_name('Expected method name after .').value
arguments: list[tuple] = []
if self._match(TokenType.LPAREN):
if not self._check(TokenType.RPAREN):
arguments = self._parse_decorator_args()
self._expect(TokenType.RPAREN, "Expected ')' after decorator arguments")
return Decorator(name=name, arguments=arguments, object=obj, location=loc)
def _parse_decorator_args(self) -> list[tuple]:
args = []
while True:
if self._check(TokenType.IDENTIFIER) and self._peek().type in (
TokenType.ASSIGN, TokenType.COLON):
key = self._advance().value
self._advance() # consume = or :
val = self.parse_expression()
args.append((key, val))
else:
args.append((None, self.parse_expression()))
if not self._match(TokenType.COMMA):
break
return args
def _parse_function(self, decorators: list[Decorator] = None) -> FunctionDecl:
loc = self._loc()
self._expect(TokenType.FUNC)
name = self._expect_name('Expected function name').value
self._expect(TokenType.LPAREN, "Expected '(' after function name")
params = self._parse_parameters()
self._expect(TokenType.RPAREN, "Expected ')'")
return_type = None
if self._check(TokenType.COLON):
return_type = self._parse_type_annotation()
self._skip_newlines()
body = self._parse_block()
return FunctionDecl(
name=name, params=params, return_type=return_type,
body=body, decorators=decorators or [], location=loc,
)
def _parse_parameters(self) -> list[Parameter]:
params: list[Parameter] = []
if self._check(TokenType.RPAREN):
return params
while True:
p_loc = self._loc()
p_name = self._expect(TokenType.IDENTIFIER, 'Expected parameter name').value
p_ann = None
p_var = False
p_def = None
if self._check(TokenType.COLON):
p_ann = self._parse_type_annotation()
if self._match(TokenType.DOTDOTDOT):
p_var = True
elif self._match(TokenType.ASSIGN):
p_def = self.parse_expression()
params.append(Parameter(
name=p_name, type_annotation=p_ann,
default=p_def, is_variadic=p_var, location=p_loc,
))
if not self._match(TokenType.COMMA):
break
return params
def _parse_class(self) -> ClassDecl:
loc = self._loc()
self._expect(TokenType.CLASS)
name = self._expect(TokenType.IDENTIFIER, 'Expected class name').value
parent = None
if self._match(TokenType.COLON):
parent = self._expect(TokenType.IDENTIFIER, 'Expected parent class name').value
self._skip_newlines()
self._expect(TokenType.LBRACE, "Expected '{'")
self._skip_newlines()
fields: list[ClassField] = []
constructor = None
methods: list[FunctionDecl] = []
while not self._check(TokenType.RBRACE, TokenType.EOF):
decorators: list[Decorator] = []
while self._check(TokenType.AT):
decorators.append(self._parse_decorator())
self._skip_newlines()
if self._check(TokenType.CONSTRUCT):
constructor = self._parse_constructor()
elif self._check(TokenType.FUNC):
methods.append(self._parse_function(decorators))
elif self._check(TokenType.IDENTIFIER):
f_loc = self._loc()
f_name = self._advance().value
f_ann = self._parse_type_annotation() if self._check(TokenType.COLON) else None
f_def = self.parse_expression() if self._match(TokenType.ASSIGN) else None
fields.append(ClassField(name=f_name, type_annotation=f_ann,
default=f_def, location=f_loc))
else:
self._error(f'Unexpected token in class body: {self._cur().type.name}')
self._advance()
self._skip_newlines()
self._expect(TokenType.RBRACE, "Expected '}'")
return ClassDecl(name=name, parent=parent, fields=fields,
constructor=constructor, methods=methods, location=loc)
def _parse_constructor(self) -> ConstructorDecl:
loc = self._loc()
self._expect(TokenType.CONSTRUCT)
self._expect(TokenType.LPAREN, "Expected '('")
params = self._parse_parameters()
self._expect(TokenType.RPAREN, "Expected ')'")
self._skip_newlines()
body = self._parse_block()
return ConstructorDecl(params=params, body=body, location=loc)
def _parse_namespace(self) -> NamespaceDecl:
loc = self._loc()
self._expect(TokenType.NAMESPACE)
name = self._expect(TokenType.IDENTIFIER, 'Expected namespace name').value
self._skip_newlines()
self._expect(TokenType.LBRACE, "Expected '{'")
self._skip_newlines()
stmts: list = []
while not self._check(TokenType.RBRACE, TokenType.EOF):
stmt = self._parse_declaration()
if stmt:
stmts.append(stmt)
self._skip_newlines()
self._expect(TokenType.RBRACE, "Expected '}'")
return NamespaceDecl(name=name, statements=stmts, location=loc)
def _parse_using(self) -> UsingStmt:
loc = self._loc()
self._expect(TokenType.USING)
name = self._expect(TokenType.IDENTIFIER, 'Expected namespace name').value
alias = None
if self._match(TokenType.ASSIGN):
alias = name
name = self._expect(TokenType.IDENTIFIER, 'Expected namespace name after =').value
names = None
if self._match(TokenType.LBRACE):
names = []
while not self._check(TokenType.RBRACE, TokenType.EOF):
names.append(self._expect(TokenType.IDENTIFIER, 'Expected symbol name').value)
if not self._match(TokenType.COMMA):
break
self._expect(TokenType.RBRACE, "Expected '}'")
return UsingStmt(namespace=name, alias=alias, names=names, location=loc)
def _parse_busing(self) -> BusingStmt:
loc = self._loc()
self._expect(TokenType.BUSING)
if self._check(TokenType.STRING):
tok = self._advance()
path = self._token_raw_str(tok)
return BusingStmt(name=None, path=path, location=loc)
name = self._expect(TokenType.IDENTIFIER, 'Expected name or path').value
self._expect(TokenType.ASSIGN, "Expected '='")
tok = self._expect(TokenType.STRING, 'Expected path string')
path = self._token_raw_str(tok)
return BusingStmt(name=name, path=path, location=loc)
# ------------------------------------------------------------------
# Statements
# ------------------------------------------------------------------
def _parse_statement(self) -> Optional[Statement]:
if self._check(TokenType.RETURN): return self._parse_return()
if self._check(TokenType.BREAK): return self._parse_break()
if self._check(TokenType.CONTINUE): return self._parse_continue()
if self._check(TokenType.IF): return self._parse_if()
if self._check(TokenType.WHILE): return self._parse_while()
if self._check(TokenType.FOR): return self._parse_for()
if self._check(TokenType.FOREACH): return self._parse_foreach()
if self._check(TokenType.WITH): return self._parse_with()
if self._check(TokenType.TRY): return self._parse_try()
if self._check(TokenType.THROW): return self._parse_throw()
if self._check(TokenType.DEFER): return self._parse_defer()
if self._check(TokenType.AWAIT): return self._parse_await()
if self._check(TokenType.ON): return self._parse_on_signal()
if self._check(TokenType.WHEN): return self._parse_when()
if self._check(TokenType.LBRACE): return self._parse_block()
return self._parse_expression_statement()
def _parse_block(self) -> Block:
loc = self._loc()
self._expect(TokenType.LBRACE, "Expected '{'")
self._skip_newlines()
stmts: list = []
while not self._check(TokenType.RBRACE, TokenType.EOF):
try:
stmt = self._parse_declaration()
if stmt:
stmts.append(stmt)
except Exception as exc:
# Last-resort recovery: record error, sync to next statement
self._error(f'Parse error: {exc}')
self._sync_to(_SYNC_STMT)
stmts.append(ErrorNode(
message=str(exc),
location=loc,
))
self._skip_newlines()
self._expect(TokenType.RBRACE, "Expected '}'")
return Block(statements=stmts, location=loc)
def _parse_return(self) -> ReturnStmt:
loc = self._loc()
self._expect(TokenType.RETURN)
value = None
if not self._check(TokenType.NEWLINE, TokenType.RBRACE, TokenType.EOF):
value = self.parse_expression()
return ReturnStmt(value=value, location=loc)
def _parse_break(self) -> BreakStmt:
loc = self._loc()
self._expect(TokenType.BREAK)
return BreakStmt(location=loc)
def _parse_continue(self) -> ContinueStmt:
loc = self._loc()
self._expect(TokenType.CONTINUE)
return ContinueStmt(location=loc)
def _parse_if(self) -> IfStmt:
loc = self._loc()
self._expect(TokenType.IF)
condition = self.parse_expression()
self._skip_newlines()
then_branch = self._parse_block()
elif_branches: list[tuple] = []
else_branch = None
self._skip_newlines()
while self._check(TokenType.ELSE):
self._advance()
if self._match(TokenType.IF):
cond = self.parse_expression()
self._skip_newlines()
block = self._parse_block()
elif_branches.append((cond, block))
self._skip_newlines()
else:
self._skip_newlines()
else_branch = self._parse_block()
break
return IfStmt(condition=condition, then_branch=then_branch,
elif_branches=elif_branches, else_branch=else_branch,
location=loc)
def _parse_while(self) -> WhileStmt:
loc = self._loc()
self._expect(TokenType.WHILE)
condition = self.parse_expression()
self._skip_newlines()
body = self._parse_block()
return WhileStmt(condition=condition, body=body, location=loc)
def _parse_for(self) -> ForStmt:
loc = self._loc()
self._expect(TokenType.FOR)
variable = self._expect(TokenType.IDENTIFIER, 'Expected loop variable').value
self._expect(TokenType.IN, "Expected 'in'")
iterable = self.parse_expression()
self._skip_newlines()
body = self._parse_block()
return ForStmt(variable=variable, iterable=iterable, body=body, location=loc)
def _parse_foreach(self) -> ForeachStmt:
loc = self._loc()
self._expect(TokenType.FOREACH)
variables = [self._expect(TokenType.IDENTIFIER, 'Expected loop variable').value]
if self._match(TokenType.COMMA):
variables.append(self._expect(TokenType.IDENTIFIER, 'Expected second variable').value)
self._expect(TokenType.IN, "Expected 'in'")
iterable = self.parse_expression()
self._skip_newlines()
body = self._parse_block()
return ForeachStmt(variables=variables, iterable=iterable, body=body, location=loc)
def _parse_with(self) -> WithStmt:
loc = self._loc()
self._expect(TokenType.WITH)
# Support two syntaxes:
# with v1, v2 in expr1, expr2 { } (all vars before single 'in')
# with v1 in expr1, v2 in expr2 { } (alternating var in expr pairs)
first_var = self._expect(TokenType.IDENTIFIER, 'Expected variable name').value
if self._check(TokenType.COMMA) and self._peek_has_in_before_lbrace():
# "with v1, v2, ... in e1, e2, ..." form
variables = [first_var]
while self._match(TokenType.COMMA):
variables.append(self._expect(TokenType.IDENTIFIER, 'Expected variable name').value)
self._expect(TokenType.IN, "Expected 'in'")
resources = [self.parse_expression()]
while self._match(TokenType.COMMA):
resources.append(self.parse_expression())
elif self._check(TokenType.IN):
# "with v1 in e1, v2 in e2, ..." form (or single resource)
variables = [first_var]
resources = []
self._expect(TokenType.IN, "Expected 'in'")
resources.append(self.parse_expression())
while self._match(TokenType.COMMA):
if self._check(TokenType.IDENTIFIER) and self._peek_ahead_is_in():
variables.append(self._advance().value)
self._expect(TokenType.IN, "Expected 'in'")
resources.append(self.parse_expression())
else:
resources.append(self.parse_expression())
else:
variables = [first_var]
self._expect(TokenType.IN, "Expected 'in'")
resources = [self.parse_expression()]
if len(variables) != len(resources):
self._error("Number of variables must match resources in 'with'")
self._skip_newlines()
body = self._parse_block()
return WithStmt(variables=variables, resources=resources, body=body, location=loc)
def _peek_has_in_before_lbrace(self) -> bool:
"""Scan forward from current position: return True if IN appears before LBRACE."""
pos = self.pos
depth = 0
while pos < len(self.tokens):
tt = self.tokens[pos].type
if tt == TokenType.LBRACE and depth == 0:
return False
if tt == TokenType.LPAREN:
depth += 1
elif tt == TokenType.RPAREN:
depth -= 1
elif tt == TokenType.IN and depth == 0:
return True
elif tt == TokenType.EOF:
return False
pos += 1
return False
def _peek_ahead_is_in(self) -> bool:
"""Return True if the token after the current one is IN."""
pos = self._pos
if pos + 1 < len(self._tokens):
return self._tokens[pos + 1].type == TokenType.IN
return False
def _parse_try(self) -> TryStmt:
loc = self._loc()
self._expect(TokenType.TRY)
self._skip_newlines()
try_block = self._parse_block()
except_clauses: list = []
finally_block = None
self._skip_newlines()
while self._match(TokenType.EXCEPT):
exc_type = exc_var = None
if self._check(TokenType.IDENTIFIER):
first = self._advance().value
if self._check(TokenType.IDENTIFIER):
exc_type = first
exc_var = self._advance().value
else:
exc_var = first
self._skip_newlines()
exc_block = self._parse_block()
except_clauses.append((exc_type, exc_var, exc_block))
self._skip_newlines()
if self._match(TokenType.FINALLY):
self._skip_newlines()
finally_block = self._parse_block()
return TryStmt(try_block=try_block, except_clauses=except_clauses,
finally_block=finally_block, location=loc)
def _parse_throw(self) -> ThrowStmt:
loc = self._loc()
self._expect(TokenType.THROW)
return ThrowStmt(expression=self.parse_expression(), location=loc)
def _parse_defer(self) -> DeferStmt:
loc = self._loc()
self._expect(TokenType.DEFER)
return DeferStmt(expression=self.parse_expression(), location=loc)
def _parse_await(self) -> AwaitStmt:
loc = self._loc()
self._expect(TokenType.AWAIT)
return AwaitStmt(expression=self.parse_expression(), location=loc)
def _parse_on_signal(self) -> OnSignalStmt:
loc = self._loc()
self._expect(TokenType.ON)
signal = self._expect(TokenType.IDENTIFIER, 'Expected signal name').value
self._skip_newlines()
body = self._parse_block()
return OnSignalStmt(signal=signal, body=body, location=loc)
def _parse_when(self) -> WhenStmt:
loc = self._loc()
self._expect(TokenType.WHEN)
value = self.parse_expression()
self._skip_newlines()
self._expect(TokenType.LBRACE, "Expected '{'")
self._skip_newlines()
branches: list[WhenBranch] = []
while not self._check(TokenType.RBRACE, TokenType.EOF):
branches.append(self._parse_when_branch())
self._skip_newlines()
self._expect(TokenType.RBRACE, "Expected '}'")
return WhenStmt(value=value, branches=branches, location=loc)
def _parse_when_branch(self) -> WhenBranch:
loc = self._loc()
if self._match(TokenType.ELSE):
self._skip_newlines()
return WhenBranch(patterns=[], is_else=True,
body=self._parse_block(), location=loc)
patterns = []
while True:
left = self._parse_primary()
if self._match(TokenType.DOTDOT):
right = self._parse_primary()
patterns.append(RangePattern(start=left, end=right, location=loc))
else:
patterns.append(left)
if not self._match(TokenType.COMMA):
break
self._skip_newlines()
return WhenBranch(patterns=patterns, is_else=False,
body=self._parse_block(), location=loc)
def _parse_expression_statement(self) -> Optional[Statement]:
loc = self._loc()
expr = self.parse_expression()
type_annotation = None
if isinstance(expr, Identifier) and self._check(TokenType.COLON):
type_annotation = self._parse_type_annotation()
if self._check(TokenType.ASSIGN, TokenType.PLUS_ASSIGN,
TokenType.MINUS_ASSIGN, TokenType.STAR_ASSIGN,
TokenType.SLASH_ASSIGN):
op = self._advance().value
value = self.parse_expression()
return Assignment(target=expr, type_annotation=type_annotation,
operator=op, value=value, location=loc)
return ExpressionStmt(expression=expr, location=loc)
# ------------------------------------------------------------------
# Expressions
# ------------------------------------------------------------------
PRECEDENCE = {
TokenType.PIPE: 0,
TokenType.OR: 1,
TokenType.AND: 2,
TokenType.EQ: 3, TokenType.NEQ: 3,
TokenType.LT: 4, TokenType.GT: 4,
TokenType.LTE: 4, TokenType.GTE: 4,
TokenType.PLUS: 5, TokenType.MINUS: 5, TokenType.DOTDOT: 5,
TokenType.STAR: 6, TokenType.SLASH: 6, TokenType.PERCENT: 6,
}
def parse_expression(self, min_prec: int = 0) -> Expression:
left = self._parse_unary()
while True:
prec = self.PRECEDENCE.get(self._cur().type, -1)
if prec < min_prec:
break
op = self._advance()
right = self.parse_expression(prec + 1)
left = BinaryOp(left=left, operator=op.value, right=right,
location=left.location)
return left
def _parse_unary(self) -> Expression:
if self._check(TokenType.NOT, TokenType.MINUS):
loc = self._loc()
op = self._advance()
return UnaryOp(operator=op.value, operand=self._parse_unary(), location=loc)
if self._check(TokenType.ASYNC):
loc = self._loc()
self._advance()
return AsyncExpr(expression=self._parse_unary(), location=loc)
if self._check(TokenType.NEW):
return self._parse_new()
return self._parse_postfix()
def _parse_new(self) -> NewExpr:
loc = self._loc()
self._advance() # consume 'new'
class_name = self._expect(TokenType.IDENTIFIER, "Expected class name after 'new'").value
self._expect(TokenType.LPAREN, "Expected '('")
args = [] if self._check(TokenType.RPAREN) else self._parse_arguments()
self._expect(TokenType.RPAREN, "Expected ')'")
return NewExpr(class_name=class_name, arguments=args, location=loc)
def _parse_postfix(self) -> Expression:
expr = self._parse_primary()
while True:
if self._match(TokenType.LPAREN):
args = [] if self._check(TokenType.RPAREN) else self._parse_arguments()
self._expect(TokenType.RPAREN, "Expected ')'")
expr = CallExpr(callee=expr, arguments=args, location=expr.location)
elif self._match(TokenType.DOT):
member = self._expect_name('Expected member name').value
expr = MemberAccess(object=expr, member=member, location=expr.location)
elif self._match(TokenType.LBRACKET):
index = self.parse_expression()
self._expect(TokenType.RBRACKET, "Expected ']'")
expr = IndexAccess(object=expr, index=index, location=expr.location)
else:
break
return expr
def _parse_arguments(self) -> list[Expression]:
args = []
while True:
args.append(self.parse_expression())
if not self._match(TokenType.COMMA):
break
return args
def _parse_primary(self) -> Expression:
loc = self._loc()
tok = self._cur()
if self._match(TokenType.INTEGER):
return IntegerLiteral(value=tok.value, location=loc)
if self._match(TokenType.FLOAT):
return FloatLiteral(value=tok.value, location=loc)
if self._match(TokenType.STRING):
return self._parse_string_literal(tok)
if self._match(TokenType.TRUE):
return BoolLiteral(value=True, location=loc)
if self._match(TokenType.FALSE):
return BoolLiteral(value=False, location=loc)
if self._match(TokenType.NIL):
return NilLiteral(location=loc)
if self._match(TokenType.THIS):
return ThisExpr(location=loc)
if self._check(TokenType.BASE):
return self._parse_base_call()
if self._match(TokenType.LBRACKET):
elements: list[Expression] = []
if not self._check(TokenType.RBRACKET):
while True:
elements.append(self.parse_expression())
if not self._match(TokenType.COMMA):
break
self._expect(TokenType.RBRACKET, "Expected ']'")
return ArrayLiteral(elements=elements, location=loc)
if self._check(TokenType.LBRACE):
return self._parse_dict_literal()
if self._check(TokenType.LPAREN):
return self._parse_paren_or_lambda()
if self._check(TokenType.IDENTIFIER):
if self._peek().type == TokenType.ARROW:
return self._parse_single_param_lambda()
self._advance()
return Identifier(name=tok.value, location=loc)
if self._check(TokenType.RANGE):
self._advance()
self._expect(TokenType.LPAREN, "Expected '('")
args = self._parse_arguments()
self._expect(TokenType.RPAREN, "Expected ')'")
return CallExpr(callee=Identifier(name='range', location=loc),
arguments=args, location=loc)
# Error recovery: record error and return a placeholder
self._error(f'Unexpected token: {tok.type.name}')
self._advance()
return ErrorNode(message=f'Unexpected {tok.type.name}', location=loc) # type: ignore
def _parse_base_call(self) -> BaseCall:
loc = self._loc()
self._expect(TokenType.BASE)
self._expect(TokenType.LPAREN, "Expected '('")
args = [] if self._check(TokenType.RPAREN) else self._parse_arguments()
self._expect(TokenType.RPAREN, "Expected ')'")
return BaseCall(arguments=args, location=loc)
def _parse_dict_literal(self) -> DictLiteral:
loc = self._loc()
self._expect(TokenType.LBRACE)
self._skip_newlines()
pairs: list[tuple] = []
while not self._check(TokenType.RBRACE, TokenType.EOF):
key = self.parse_expression()
self._expect(TokenType.COLON, "Expected ':' after dict key")
val = self.parse_expression()
pairs.append((key, val))
self._skip_newlines()
if not self._match(TokenType.COMMA):
break
self._skip_newlines()
self._skip_newlines()
self._expect(TokenType.RBRACE, "Expected '}'")
return DictLiteral(pairs=pairs, location=loc)
def _parse_paren_or_lambda(self) -> Expression:
"""Parse either a parenthesised expression or a multi-param lambda."""
loc = self._loc()
self._expect(TokenType.LPAREN)
# () => (zero-param lambda)
if self._check(TokenType.RPAREN):
self._advance()
if self._match(TokenType.ARROW):
return self._parse_lambda_body([], loc)
self._error("Expected '=>' after '()'")
return NilLiteral(location=loc)
# Speculative: try to parse (ident, ident, ...) =>
if self._check(TokenType.IDENTIFIER):
saved = self.pos
params: list[str] = []
ok = True
try:
while True:
if not self._check(TokenType.IDENTIFIER):
ok = False; break
params.append(self._advance().value)
if not self._match(TokenType.COMMA):
break
if ok and self._match(TokenType.RPAREN) and self._match(TokenType.ARROW):
return self._parse_lambda_body(params, loc)
except Exception:
pass
self.pos = saved
# Regular parenthesised expression
expr = self.parse_expression()
self._expect(TokenType.RPAREN, "Expected ')'")
return expr
def _parse_single_param_lambda(self) -> Lambda:
"""Parse `ident => body`."""
loc = self._loc()
param = self._expect(TokenType.IDENTIFIER).value
self._expect(TokenType.ARROW)
return self._parse_lambda_body([param], loc)
def _parse_lambda_body(self, param_names: list[str], loc: SourceLocation) -> Lambda:
params = [Parameter(name=n, location=loc) for n in param_names]
body = self._parse_block() if self._check(TokenType.LBRACE) else self.parse_expression()
return Lambda(params=params, body=body, location=loc)
# ------------------------------------------------------------------
# Type annotations
# ------------------------------------------------------------------
def _parse_type_annotation(self) -> Optional[TypeAnnotation]:
if not self._match(TokenType.COLON):
return None
return self._parse_type()
def _parse_type(self) -> TypeAnnotation:
loc = self._loc()
if self._check(TokenType.LPAREN):
return self._parse_function_type(loc)
name = self._expect(TokenType.IDENTIFIER, 'Expected type name').value
if self._match(TokenType.LBRACKET):
if name == 'dict':
key_type = self._parse_type()
self._expect(TokenType.COMMA, "Expected ','")
val_type = self._parse_type()
self._expect(TokenType.RBRACKET, "Expected ']'")
return TypeAnnotation(name='dict', key_type=key_type,
value_type=val_type, location=loc)
self._expect(TokenType.RBRACKET, "Expected ']'")
return TypeAnnotation(name=name, is_array=True, location=loc)
return TypeAnnotation(name=name, location=loc)
def _parse_function_type(self, loc: SourceLocation) -> TypeAnnotation:
self._expect(TokenType.LPAREN, "Expected '('")
param_types: list[TypeAnnotation] = []
if not self._check(TokenType.RPAREN):
while True:
param_types.append(self._parse_type())
if not self._match(TokenType.COMMA):
break
self._expect(TokenType.RPAREN, "Expected ')'")
self._expect(TokenType.ARROW, "Expected '=>'")
return_type = self._parse_type()
return TypeAnnotation(name='func', param_types=param_types,
return_type=return_type, location=loc)
# ---------------------------------------------------------------------------
# Convenience function
# ---------------------------------------------------------------------------
def parse(source: str, filename: str = '<stdin>') -> tuple[Program, list[str]]:
"""Lex + parse source, return (Program, errors)."""
from ..lexer.lexer import tokenize as lex_tokenize
tokens, lex_errors = lex_tokenize(source, filename)
lex_msgs = [str(e) for e in lex_errors]
p = Parser(tokens, filename)
prog = p.parse()
return prog, lex_msgs + p.errors
content
View file @ 31bfe20c
#!/usr/bin/env python3 #!/usr/bin/env python3
"""
ContenT Compiler Entry Point
"""
import sys import sys
import os import os
# Add bootstrap to path
sys.path.insert (0, os.path.dirname (os.path.abspath (__file__))) sys.path.insert (0, os.path.dirname (os.path.abspath (__file__)))
from bootstrap.main import main from compiler.cli import main
if __name__ == "__main__": if __name__ == "__main__":
sys.exit (main ()) sys.exit (main ())
content.in
View file @ 31bfe20c
#!@PYTHON@ #!@PYTHON@
import sys import sys
sys.path.insert(0, '@pkgdatadir@') sys.path.insert(0, '@pkgdatadir@')
from bootstrap.main import main from compiler.cli import main
if __name__ == "__main__": if __name__ == "__main__":
sys.exit(main()) sys.exit(main())
meson.build
View file @ 31bfe20c
...@@ -8,7 +8,7 @@ py = python.find_installation('python3') ...@@ -8,7 +8,7 @@ py = python.find_installation('python3')
pkgdatadir = get_option('prefix') / get_option('datadir') / meson.project_name() pkgdatadir = get_option('prefix') / get_option('datadir') / meson.project_name()
subdir('bootstrap') subdir('compiler')
subdir('lib') subdir('lib')
test('pytest', py, test('pytest', py,
......
tests/conftest.py
View file @ 31bfe20c
...@@ -5,9 +5,9 @@ sys.path.insert(0, str(Path(__file__).parent.parent)) ...@@ -5,9 +5,9 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
sys.path.insert(0, str(Path(__file__).parent)) sys.path.insert(0, str(Path(__file__).parent))
import pytest import pytest
from bootstrap.lexer import Lexer from compiler.lexer.lexer import Lexer
from bootstrap.parser import Parser from compiler.syntax.parser import parse as _parse_source
from bootstrap.codegen import CodeGenerator from compiler.cli import compile_source
@pytest.fixture @pytest.fixture
...@@ -21,20 +21,14 @@ def lex(): ...@@ -21,20 +21,14 @@ def lex():
@pytest.fixture @pytest.fixture
def parse(): def parse():
def _parse(source: str): def _parse(source: str):
lexer = Lexer(source) ast, errors = _parse_source(source, '<test>')
tokens = lexer.tokenize() return ast
parser = Parser(tokens)
return parser.parse()
return _parse return _parse
@pytest.fixture @pytest.fixture
def compile_ct(): def compile_ct():
def _compile(source: str) -> str: def _compile(source: str) -> str:
lexer = Lexer(source) ok, script, errs = compile_source(source, '<test>')
tokens = lexer.tokenize() return script
parser = Parser(tokens)
ast = parser.parse()
gen = CodeGenerator()
return gen.generate(ast)
return _compile return _compile
tests/helpers.py
View file @ 31bfe20c
import subprocess import subprocess
import tempfile import tempfile
import os import os
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent.parent))
def run_ct(source: str) -> tuple[int, str, str]: from compiler.cli import compile_source
with tempfile.NamedTemporaryFile(mode='w', suffix='.ct', delete=False) as f:
f.write(source)
f.flush() def run_ct(source: str, timeout: int = 10) -> tuple[int, str, str]:
ct_file = f.name rc, script, errs = _compile(source)
if rc != 0:
return rc, '', errs
with tempfile.NamedTemporaryFile('w', suffix='.sh', delete=False) as f:
f.write(script)
fname = f.name
try: try:
result = subprocess.run( result = subprocess.run(
['python3', 'content', 'run', ct_file], ['bash', fname], capture_output=True, text=True, timeout=timeout
capture_output=True,
text=True,
timeout=10
) )
return result.returncode, result.stdout, result.stderr return result.returncode, result.stdout, result.stderr
finally: finally:
os.unlink(ct_file) os.unlink(fname)
def compile_ct(source: str) -> tuple[int, str, str]: def compile_ct(source: str) -> tuple[int, str, str]:
with tempfile.NamedTemporaryFile(mode='w', suffix='.ct', delete=False) as f: return _compile(source)
f.write(source)
f.flush()
ct_file = f.name
sh_file = ct_file.replace('.ct', '.sh')
try:
result = subprocess.run(
['python3', 'content', 'build', ct_file, '-o', sh_file],
capture_output=True,
text=True,
timeout=10
)
compiled_output = ""
if os.path.exists(sh_file):
with open(sh_file, 'r') as sf:
compiled_output = sf.read()
os.unlink(sh_file)
return result.returncode, compiled_output, result.stderr
finally:
os.unlink(ct_file)
def compile_ct_with_flags(source: str, flags: list = None) -> tuple[int, str, str]: def compile_ct_with_flags(source: str, flags: list = None) -> tuple[int, str, str]:
...@@ -72,25 +56,8 @@ def compile_ct_with_flags(source: str, flags: list = None) -> tuple[int, str, st ...@@ -72,25 +56,8 @@ def compile_ct_with_flags(source: str, flags: list = None) -> tuple[int, str, st
def compile_ct_check(source: str) -> tuple[int, str, str]: def compile_ct_check(source: str) -> tuple[int, str, str]:
with tempfile.NamedTemporaryFile(mode='w', suffix='.ct', delete=False) as f: rc, _, err = _compile(source)
f.write(source) return rc, '', err
f.flush()
ct_file = f.name
sh_file = ct_file.replace('.ct', '.sh')
try:
result = subprocess.run(
['python3', 'content', 'build', ct_file, '-o', sh_file],
capture_output=True,
text=True,
timeout=10
)
if os.path.exists(sh_file):
os.unlink(sh_file)
return result.returncode, result.stdout, result.stderr
finally:
os.unlink(ct_file)
def run_ct_test(source: str) -> tuple[int, str, str]: def run_ct_test(source: str) -> tuple[int, str, str]:
...@@ -109,3 +76,13 @@ def run_ct_test(source: str) -> tuple[int, str, str]: ...@@ -109,3 +76,13 @@ def run_ct_test(source: str) -> tuple[int, str, str]:
return result.returncode, result.stdout, result.stderr return result.returncode, result.stdout, result.stderr
finally: finally:
os.unlink(ct_file) os.unlink(ct_file)
def _compile(source: str) -> tuple[int, str, str]:
try:
ok, script, errs = compile_source(source, '<test>')
if not ok:
return 1, '', '\n'.join(str(e) for e in errs)
return 0, script, ''
except Exception as e:
return 1, '', str(e)
tests/test_autoscan.py
View file @ 31bfe20c
import os import os
import tempfile import tempfile
import shutil import shutil
from bootstrap.main import find_ct_files, compile_files from compiler.cli import find_ct_files, compile_files
from helpers import run_ct from helpers import run_ct
......
tests/test_build_lib.py
View file @ 31bfe20c
...@@ -14,7 +14,7 @@ class TestBuildLib: ...@@ -14,7 +14,7 @@ class TestBuildLib:
sh_path = ct_path.replace(".ct", ".sh") sh_path = ct_path.replace(".ct", ".sh")
try: try:
result = subprocess.run( result = subprocess.run(
[sys.executable, "-m", "bootstrap.main", "build-lib", ct_path], [sys.executable, "-m", "compiler", "build-lib", ct_path],
capture_output=True, text=True, capture_output=True, text=True,
cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))) cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
) )
...@@ -42,7 +42,7 @@ class TestBuildLib: ...@@ -42,7 +42,7 @@ class TestBuildLib:
try: try:
result = subprocess.run( result = subprocess.run(
[sys.executable, "-m", "bootstrap.main", "build-lib", ct_path, "-o", out_path], [sys.executable, "-m", "compiler", "build-lib", ct_path, "-o", out_path],
capture_output=True, text=True, capture_output=True, text=True,
cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))) cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
) )
...@@ -67,7 +67,7 @@ class TestBuildLib: ...@@ -67,7 +67,7 @@ class TestBuildLib:
out_path = os.path.join(d, "out.sh") out_path = os.path.join(d, "out.sh")
result = subprocess.run( result = subprocess.run(
[sys.executable, "-m", "bootstrap.main", "build-lib", d, "-o", out_path], [sys.executable, "-m", "compiler", "build-lib", d, "-o", out_path],
capture_output=True, text=True, capture_output=True, text=True,
cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))) cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
) )
...@@ -87,7 +87,7 @@ class TestBuildLib: ...@@ -87,7 +87,7 @@ class TestBuildLib:
sh_path = ct_path.replace(".ct", ".sh") sh_path = ct_path.replace(".ct", ".sh")
try: try:
result = subprocess.run( result = subprocess.run(
[sys.executable, "-m", "bootstrap.main", "build-lib", ct_path], [sys.executable, "-m", "compiler", "build-lib", ct_path],
capture_output=True, text=True, capture_output=True, text=True,
cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))) cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
) )
...@@ -119,7 +119,7 @@ class TestBuildLib: ...@@ -119,7 +119,7 @@ class TestBuildLib:
try: try:
result = subprocess.run( result = subprocess.run(
[sys.executable, "-m", "bootstrap.main", "build-lib", ct_path, "--install"], [sys.executable, "-m", "compiler", "build-lib", ct_path, "--install"],
capture_output=True, text=True, capture_output=True, text=True,
cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))) cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
) )
......
tests/test_busing.py
View file @ 31bfe20c
...@@ -2,19 +2,13 @@ import pytest ...@@ -2,19 +2,13 @@ import pytest
import os import os
import tempfile import tempfile
from helpers import run_ct, compile_ct from helpers import run_ct, compile_ct
from bootstrap.lexer import Lexer from compiler.syntax.nodes import BusingStmt
from bootstrap.parser import Parser from compiler.cli import compile_source as _compile_src
from bootstrap.codegen import CodeGenerator
from bootstrap.ast_nodes import BusingStmt
def compile_source(source): def compile_source(source):
lexer = Lexer(source) ok, script, errs = _compile_src(source, '<test>')
tokens = lexer.tokenize() return script
parser = Parser(tokens)
ast = parser.parse()
gen = CodeGenerator()
return gen.generate(ast)
class TestBusingParser: class TestBusingParser:
......
tests/test_decorators.py
View file @ 31bfe20c
import pytest
from helpers import run_ct, compile_ct, compile_ct_with_flags, run_ct_test from helpers import run_ct, compile_ct, compile_ct_with_flags, run_ct_test
......
tests/test_lexer.py
View file @ 31bfe20c
import pytest import pytest
from bootstrap.lexer import Lexer from compiler.lexer.lexer import Lexer
from bootstrap.tokens import TokenType from compiler.lexer.tokens import TokenType, RawText, RawInterp
class TestLexerBasics: class TestLexerBasics:
...@@ -43,25 +43,28 @@ class TestLexerStrings: ...@@ -43,25 +43,28 @@ class TestLexerStrings:
def test_simple_string(self, lex): def test_simple_string(self, lex):
tokens = lex('"hello"') tokens = lex('"hello"')
assert tokens[0].type == TokenType.STRING assert tokens[0].type == TokenType.STRING
assert tokens[0].value == "hello" assert tokens[0].value == [RawText("hello")]
def test_single_quotes(self, lex): def test_single_quotes(self, lex):
tokens = lex("'world'") tokens = lex("'world'")
assert tokens[0].type == TokenType.STRING assert tokens[0].type == TokenType.STRING
assert tokens[0].value == "world" assert tokens[0].value == [RawText("world")]
def test_escape_newline(self, lex): def test_escape_newline(self, lex):
tokens = lex(r'"line1\nline2"') tokens = lex(r'"line1\nline2"')
assert tokens[0].value == "line1\nline2" assert tokens[0].value == [RawText("line1\nline2")]
def test_escape_tab(self, lex): def test_escape_tab(self, lex):
tokens = lex(r'"col1\tcol2"') tokens = lex(r'"col1\tcol2"')
assert tokens[0].value == "col1\tcol2" assert tokens[0].value == [RawText("col1\tcol2")]
def test_escape_braces(self, lex): def test_escape_braces(self, lex):
tokens = lex(r'"literal \{ brace \}"') tokens = lex(r'"literal \{ brace \}"')
assert "\x00LBRACE\x00" in tokens[0].value parts = tokens[0].value
assert "\x00RBRACE\x00" in tokens[0].value assert len(parts) == 1
assert isinstance(parts[0], RawText)
assert "{" in parts[0].text
assert "}" in parts[0].text
class TestLexerIdentifiers: class TestLexerIdentifiers:
......
tests/test_namespace.py
View file @ 31bfe20c
import pytest import pytest
from helpers import run_ct, compile_ct from helpers import run_ct, compile_ct
from bootstrap.lexer import Lexer from compiler.syntax.nodes import NamespaceDecl, UsingStmt, FunctionDecl, ClassDecl
from bootstrap.parser import Parser from compiler.cli import compile_source as _compile_src
from bootstrap.codegen import CodeGenerator
from bootstrap.ast_nodes import NamespaceDecl, UsingStmt, FunctionDecl, ClassDecl
def compile_source(source): def compile_source(source):
lexer = Lexer(source) ok, script, errs = _compile_src(source, '<test>')
tokens = lexer.tokenize() return script
parser = Parser(tokens)
ast = parser.parse()
gen = CodeGenerator()
return gen.generate(ast)
class TestNamespaceParser: class TestNamespaceParser:
...@@ -83,8 +77,8 @@ namespace models { ...@@ -83,8 +77,8 @@ namespace models {
u = models.User("Alice") u = models.User("Alice")
''' '''
code = compile_source(source) code = compile_source(source)
assert 'models__User ()' in code assert 'models__User' in code
assert '__ct_class_models__User_construct' in code assert '__ct_class_User_construct' in code
def test_using_direct_access(self): def test_using_direct_access(self):
source = ''' source = '''
...@@ -160,22 +154,9 @@ using utils ...@@ -160,22 +154,9 @@ using utils
upper("test") upper("test")
lower("test") lower("test")
''' '''
lexer1 = Lexer(source1) code = compile_source(source1 + "\n" + source2)
tokens1 = lexer1.tokenize() assert 'utils__upper' in code
parser1 = Parser(tokens1) assert 'utils__lower' in code
ast1 = parser1.parse()
lexer2 = Lexer(source2)
tokens2 = lexer2.tokenize()
parser2 = Parser(tokens2)
ast2 = parser2.parse()
gen = CodeGenerator()
code = gen.generate_multi([ast1, ast2])
assert 'utils__upper ()' in code
assert 'utils__lower ()' in code
assert 'utils__upper "test"' in code
assert 'utils__lower "test"' in code
class TestNamespaceIntegration: class TestNamespaceIntegration:
......
tests/test_parser.py
View file @ 31bfe20c
import pytest import pytest
from bootstrap.ast_nodes import ( from compiler.syntax.nodes import (
Program, FunctionDecl, ClassDecl, ClassField, ConstructorDecl, Program, FunctionDecl, ClassDecl, ClassField, ConstructorDecl,
IntegerLiteral, FloatLiteral, StringLiteral, BoolLiteral, NilLiteral, IntegerLiteral, FloatLiteral, StringLiteral, BoolLiteral, NilLiteral,
Identifier, ArrayLiteral, DictLiteral, BinaryOp, UnaryOp, Identifier, ArrayLiteral, DictLiteral, BinaryOp, UnaryOp,
...@@ -29,7 +29,8 @@ class TestParserLiterals: ...@@ -29,7 +29,8 @@ class TestParserLiterals:
ast = parse('"hello"') ast = parse('"hello"')
stmt = ast.statements[0] stmt = ast.statements[0]
assert isinstance(stmt.expression, StringLiteral) assert isinstance(stmt.expression, StringLiteral)
assert stmt.expression.value == "hello" assert len(stmt.expression.parts) == 1
assert stmt.expression.parts[0].value == "hello"
def test_bool_true(self, parse): def test_bool_true(self, parse):
ast = parse("true") ast = parse("true")
...@@ -204,13 +205,13 @@ class TestParserLambdas: ...@@ -204,13 +205,13 @@ class TestParserLambdas:
stmt = ast.statements[0] stmt = ast.statements[0]
assert isinstance(stmt, Assignment) assert isinstance(stmt, Assignment)
assert isinstance(stmt.value, Lambda) assert isinstance(stmt.value, Lambda)
assert stmt.value.params == ["x"] assert [p.name for p in stmt.value.params] == ["x"]
def test_multi_param_lambda(self, parse): def test_multi_param_lambda(self, parse):
ast = parse("add = (a, b) => a + b") ast = parse("add = (a, b) => a + b")
stmt = ast.statements[0] stmt = ast.statements[0]
assert isinstance(stmt.value, Lambda) assert isinstance(stmt.value, Lambda)
assert stmt.value.params == ["a", "b"] assert [p.name for p in stmt.value.params] == ["a", "b"]
def test_block_lambda(self, parse): def test_block_lambda(self, parse):
ast = parse("fn = x => { return x * 2 }") ast = parse("fn = x => { return x * 2 }")
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment