Specs

How Common works inside, for the curious: the pipeline, the values, the safety model, the limits, and how fast it is. Everything here is true of version 0.4.0.

At a glance

How a page runs

1. The .html fileread whole. Text outside <cl> tags is copied to the output untouched.
2. Blockseach <cl> … </cl> block is cut into lines. Line numbers count from the top of the file, so errors point at the real line.
3. Tokensthe first time a line runs, it is split into words, numbers, text and symbols, and the tokens are kept with the line. A loop does not re-read its own lines.
4. Parse and run, togethera recursive-descent parser reads each line and evaluates it as it goes. There is no syntax tree and no bytecode: the tokens are the program.
5. Outputwhat the block showed replaces the block. Visitor values were HTML-escaped as they were shown or placed in text, exactly once.

Blocks are found by their shape before anything runs: if, for each, repeat, to, send email, a record and a text block each look ahead for the end that closes them, so a missing end is reported at the line that opened the block.

Values

Values are copied, never shared: setting one name to another gives an independent copy. When one side of a comparison is a number, text written as a number is compared as a number, so query "age" is more than 12 works on what a visitor typed. Comparing a number with other text is an error, not a quiet no.

The safety model

Every value carries two marks. Untrusted means it came straight from a visitor: query, form, path, remembered, and text read from a database. Tainted means it was built from untrusted text, through brackets, trim of, replace, split, a text block or any other word. The marks travel with the value through every word.

The server

Sessions and passwords

• Sessions. remember stores values in one cookie, cl_session: the values and an end date 30 days out, signed with HMAC-SHA256 under a 32-byte random secret kept in cl.secret. The signature is checked in constant time, and a cookie that fails is ignored. The cookie is HttpOnly and SameSite=Lax.
• Passwords. hash of makes PBKDF2-HMAC-SHA256 with 200,000 rounds and a 16-byte random salt, written cl2$200000$salt$hash. matches checks it in constant time. Hashes from 0.3.0 and earlier (cl1$) still match.
• Randomness. From /dev/urandom, or rand_s on Windows. With neither, cl stops rather than guess.
• SHA-256, HMAC and PBKDF2 are written out in cl.c: no crypto library is needed.

Databases and email

• SQLite waits up to 3 seconds for a write lock instead of failing. Postgres gets the same ? placeholders, turned into $1, $2. Numbers come back as numbers, and booleans as yes or no.
• Email goes to an outbox/ folder under cl serve, and to sendmail under cl cgi. Plain text, UTF-8, with a non-ASCII subject encoded. At most 3 a page and 20 an hour per site, a body of at most 50,000 letters, and line breaks in a subject become spaces.

Limits

Memory

There is no garbage collector. Values are copied when they are stored and freed when their name is replaced or its page, verb or loop ends. A verb reads its caller's names in place instead of copying them all, and makes its own copy only when it changes one in place (add, set … of), so the caller's values never change.

Errors are a longjmp out of whatever was running. Anything a frame was holding at that moment (token lists, values, text buffers, verb environments) is listed in small registries, and the block that catches the error frees what is left, so an error never leaks.

Speed

Common is an interpreter, and for a website the time goes to the network and the database: the pages on this site render in about 10 ms. In tight loops, the numbers below were measured on an Apple Silicon laptop with make bench. Expect different numbers on other machines; the comparison is what matters.

What made 0.4 faster, each found with a profiler rather than guessed:

• Lines are read once. Tokens are kept with their line, and a verb's body is split once rather than on every call.
• Holes are read once. The tokens of each [hole] are kept by their text in a table of 1,024 entries (by content, never by address, since text can be temporary).
• Names are checked once. "Is this a Common word?" is answered once per token, and dictionary lookups use a hash table.
• Verbs share instead of copy. Calling a verb used to copy every name on the page.
• Hints are lazy. The "you forgot the quotes" check runs only the first time a line runs, and error hints are built only when there is an error.

Not done yet: numbered slots for names instead of lookups by name, and keeping a server process alive with pages already parsed. Each would help, but neither is needed for pages this size.

Platforms

See also: how a site fits together, the glossary of every word, and the error messages.