User:Krauss/ART

...THIS PLACE IS FOR A ARXIV.ORG ARTICLE COLABORATIVE COSNTRUCTION ...

Introduction

The term template, when used in the context of digital documents and word processing, refers to a "fill-in-the-blank" document that can be completed either by hand or through an automated process. Form letters are typical example: a letter written from a template, frequently used for "spam", by marketing campaigns. It allows a mass production of very similar letters. Another classic example is the "letter frame": any content is filled in the blank of a "header and footer" frame template. It allows a mass production of non-similar content letters, with a "standard frame". Both are examples of (production of) a set of "template derived documents".

Digital documents, when used in a distribution or interchange context, like in the web (web pages as documents), have another demand. Instead of a set of documents, the web designers want a unique "adaptative" dynamic document.

In the web jargon, dynamic document is a kind of document that has been prepared with fresh or customized information, for each individual viewing. It is not static because it changes with: the time (ex. a news content), the user (ex. preferences in a login session), the user interaction (ex. web page game), the context, or all of them.

Resuming, digital documents obtained from a "templating process" have two kinds of popular products: a set of "derived documents" or a unique "dynamical document".

If the context is changed from "document" to "software source code file", we emerge with another similar aplications: template metaprogramming, macro-languages, pre-processors, documentation generators, and others. Intuitivelly we can label all of then "template systems".

This article was motivated by a set of observable problems in the context of what we will more rigorousely naming "template systems",

Conceptual problem: there are many things that we can considerate "template systems", but there are a lack of a rigourous (mathematic) definition for it.
Diversity problems: ("Babel problems"),
- observable diversty: at Wikipedia listing and comparing about 60 distint "web template engines", and another more ... not web templates...
- methodologic diversty: how to select, when use one or another type of solution
- terminologic diversty: conflics and/or ambiguities in terms as "template engine", "template processor", "style sheet", "template script", "template oriented program", etc.

Another authors, like Parr [1], X [2], Y [3], was working with these problems, and draw a lot of cues and partial solutions. We are "updating" and generalizing these author ideas.

From dynamic documents to templates

There are many ways to express dynamical documents, all of then requer documents peaces and logic peaces. This is a very simple HTML dynamic document:

<html>Hello {$x}!</html>

where {$x} is a input variable.

...

Hooks and backgrounds

... hooks ...

Document designer perseption:

/^D(sD)*s?$/: document-background template language

/^s(Ds)*D?$/: script-background template language

if the language have "ever balanced", there are no final "s?" or "t?" pattern. ....

System characterization

Motivations and methodological background

There are many systems promoted as being template systems. We need an objective criteria to select what is and what is not a template system.

To arrive at our formal definition, we first evaluate a "candidate system" and determine whether it meets a predetermined set of standards based on its input and output characteristics. For any "candidate system" to be characterized as a template system, we have to abstract it as a black box and then use Black box testing and Use cases as our methods of evaluation. This methodology ensures that our formal definition is based solely on observable elements (input and output) and thus does not rely on informally promoted definitions.

A simple template system illustration (like that below) indicates what we need to consider in our black box testing, based on specfic inputs and outputs:

The template engine, a process, is the Black box;
The template and contents are inputs;
The output documents are outputs.

To answer the question, What is and what is not a template system?, we need, for the Black box testing methodology, to fix a set of use case tests. A set of positive results on the tests must reflect the essential properties of a template system.

We use a mathematical system model to specify with precision a formal analytical framework to establish these properties.

Informal Template characterization

A simple template T is a "output document with holes", where holes are place-holders or macro references. A simple and well-accepted example of "hole" is a HTML document with a place-holder:

<html>Hello {$x}!</html>

where {$x} is a input variable. (...)

The use of *hooks* are exemplified by the red marks. It permits the separation between logic (hidden for blue designers-view) and design.
Below the source code example, where **black** and **green** are two distinct languages.

Separation of concerns necessities (e. g. content from presentation on Web templates) require a low-level separation strategy to isolate script language from output language. Then, template syntax need special care with the "border" between languages, to avoid mixing and to supply scaping forms. There are well defined tags, marks or characteres, named "hooks", that intend to separate (and compatibilize) the two languages.

Refs X, Y, Z suggest os seguintes types of hooks:

Script hooks: encloses blocks of developer-supplied program logic. Examples:
<% script %> ASP style

<? script ?> PHP style

<cfscript> script </cfscript> ColdFusion style
Sub-template hooks: to fix the frontiers of the sub-template block.
 ...  PHPlib style

<xsl:template name="stName"> ... </xsl:template> XSLT style
Expression hooks: to encode scalar variables, sub-template references, or expressions. Examples:
<%=x %> ASP style

{$x} XQuery style

<xsl:value-of select="$x" /> XSLT style

There are a hi diversity of styles for hook encoding, but a generative grammar (...) show that is a common approach.

A template T is a string that can be split (using "hook criteria") into 2 distinct, not empty, token types:

t: output document contiguous fragments.
s: script contiguous fragments, like expressions or instructions — simple instructions, or statements, or directives, or blocks of them.
Note: a sequence of repeated s, like occurs with XSLT or ColdFusion, is transformed into a unique "contiguous s" block.

The resulted sequence of tokens is not arbitrary, and, theoretically the "contiguous hypothesis" enforce a pattern that avoid validation necessity.

A rigor T is supplied by a generative grammar, $G=(N,\Sigma ,Q,A)$ , with $N=\{A,X,Y\}$ , $\Sigma =\{t,s\}$ , $A$ the start symbol, and $Q$ the following production rules: $A:=X|Y|s|t$ ; $X:=tsX|ts|tst$ ; $Y:=stY|st|sts$ .

Informal check-list of essential properties

Writing for Wikipedia community, the first author was suggested a little set of "essential properties", translated as:

An empty template supplies an empty document: the template system can not add information to the empty template.
Content independence on a template without instructions: for usual template languages, an output document file (like using an HTML file as a PHP file) is also a valid template. On other template languages we can not only copy/paste a document to the template for this proposition, we need to add hooks (and/or headers) to express a "template container", see XSLT. To generalize the idea we can use a kind of "cleaning function" to eliminate script language from the template. This cleaning function can be exemplified with a little PHP code (see fig. 1):
PHP source (all template): <html><? if ($flag){?> Hello <?= $x; ?>! <? } else { ?> Bye <? }?> </html>
Cleaned PHP (only the black portion of fig.1): <html> Hello ! Bye </html>.
The output document is always a "cleaned document", without any trace of the script language. All the "template language code" is processed by the template engine in one step.
No information is generated by the template engine.
The output documents of templates with only variations on presentation, have also only variations on presentation, independent of the content. If we compare both, they contain the same information.

-- Note: see also FAQ about use of the definitions.

Formal characterization

Elements (C,T,P,R) on the dataflow representation.

To formally define, we can modeling the (dataflow) template system as a function and your parameters:

Inputs:
- Template library, L = {T₁, T₂, … T_i, … T_N}
  There are N templates, identified by the index i. The template T₁ is also used as "default root template", when engine need a pre-defined root. For express an input with a single template (N=1) the alternative notation L={T₁}={T}, can be used.
- Content, C $\subseteq$ D
  C is a set (or a sequence) of incoming data values, from the content resource. C is in principle read-only, but engine can assign values for simplify the variable declaration feature. Alternative notation for express elements (attributes) C = {c₁, c₂, … c_j, … c_M}
  
  D is the data model, formally a (universe) set of all possible contents. It can also specify data structure — when C is not a single set of values, like a sequence (ordered set), or a XML input.
Process, P(L, C)
It is a black box model for the template engine. Suppose P as a overloading, to express singular case, P(T, C)=P({T},C). Systems that only work as a P(T, C) process — perhaps using implicit sub-templates, but not a template library — can said "lib-less" systems.
Process output, R = P(L, C)
On web template systems, R is a web document. On (generic) template systems, R is any kind of document.

Essential properties

Let:

ø: an empty content, empty template or empty document.
T_nop: a template with no programming operation (without template instructions).
L_nop = { T | T is a T_nop}
T_a, T_b are "presentation variants".
(L_a, L_b) is a library relationship where all T_{a, i} from L_a have a correspondent T_{b, j} from L_b.
Clean(T) is a "clean function" that extract all fragments (with respective hooks) of the script language.
For template languages like Haml, that specify the output language into an "alternative syntax", it is necessary that this syntax is reversible^[1] to the output language, then, the clean operator also embody this reversion.
I(X) is a "information content set" function, like a set of words from a txt converter. If X is a template, the I(X) process starts with Clean(X). If it is a library, it is applied for all library templates.
For T on the above example, I(T) = {Hello, Bye}.
P' process, returns a "expanded equivalent template". A engine (or manual procedure) that only find and expand sub-template references.

Essential properties (for all L, L_a, L_b, L_nop, and C), that template systems satisfy:

#	Property	Notes
1	P({ø},C) = ø	ø bypass (equivalence between cleaned empty template and empty output).
2	P(L_nop,ø) = P(L_nop, C) = Clean(P'(L_nop,ø))	Content independence on the "no instructions template". T_nop bypass. For lib-less systems (where L={T}), P(T_nop,ø) = Clean(T_nop).
3	P(L, C) = Clean(P(L, C))	R have nothing to process if it is used as a template, it can only be used as a T_nop.
4	I(P(L, C)) $\subseteq$ I(P'(L, C)) U I(C)	No information generated by P. For lib-less I(P(T, C)) $\subseteq$ I(T) U I(C).
5	I(P(L_a,C)) = I(P(L_b,C))	Content conservation on "presentation variants". Obs.: imply I(P'(L_a,C)) = I(P'(L_b,C)) and, For lib-less I(T_a) = I(T_b) .

Notes:

See corresponding informal properties.
Template systems modeled as complex dataflows, must use simplifier hypothesis to the characterization.
Template systems modeling is the first and fundamental step to the characterization. Example:
for characterize a documentation generator as a template system, the source code (commented or not) is modeled as (structured) content, and usually templates are internal (not customizable) to the system, the configuration files are modeled as content.
It is possible a P composition (pipeline), if the content, C, and output, R, are on the same format, like XML. Example: a composition P_XSLT(L, P_XQuery(T, C)) on Cocoon pipeline (see also XML transformation languages).
If no content on template, I(T)=ø, and script language is regular, then T is like a schema. There are cases where the template language is like a "programmable schema specification" (compare Haml with RELAX Compact_syntax).
For validate a "candidate system" we need the correspondent Clean and P', as specific checking tools, and I as a generic checking tool.

Important conclusions from definitions:

The simplest "substitution string system" can characterized as template system.
All template language need clear rules and "syntax facilities" (simple for human and machine) to evaluate the Clear(T) function.
Template files from languages like XSLT or XQuery are templates, but files like a Perl script, evaluated by a usual P_perl interpreter, that need output instruction like print and has no hook notation, are not.
The process cannot be arbitrary.
The possibility of use sub-templates is a feature for template languages, is not a general characteristic.
The possibility of sub-template recurrence relation is also a feature (may be formalized by P' but not is modeled to preserve the simplicity of template system definition).

Referential "driven types"

For specify projects and division of tasks, designers and programmers need to adopt objective point of view to see and to organize a template set, that result into a certain dichotomy of "referential types of systems". The types of system strategies are with respective P process, and how engines do decisions about sub-template choices:

Script-driven template systems:
- Designer's perception: the template engine "select template fragments and fill it with content".
- Programmer's perception: all the logic about sub-template choices (typically a if/then or switch/case logic) are explicit into the script.
- Examples: SSI (simplest lib-less), XQuery (sophisticated lib-less), Smarty (sophisticated).
- Notes: need a "root template", to express the logic for select "first level sub-templates".
Content-driven template systems:
- Designer's perception: the template engine "select desirable content and frame it with template fragments".
- Programmer's perception: part of the logic is implicit (not expressed on script), and is on engine as pre-defined rules, that permit the content choose (select) what sub-template will be used.
- Example: XSLT (lib), attribute languages like Zope (lib-less on typical uses).
- Notes: to supply the content-driven sub-template reference language feature (content choice by a dynamic context), the engine use a dispatcher or another event-driven algorithms.

There are also mixed types: a script-driven template system augmented with a kind of "match and referrer sub-template by ID" (a simple hash dispatcher can do it) instead direct control. By other hand, a content-driven template system can express traditional logic into a root template, used as a script-driven template.

Architecture characterization

The architecture of template systems, into a client-server reference model, is the main split criteria for group then. There are illustrated (see on links) three groups: Outside server systems, Server-side systems, and Distributed systems. A formal characterization avoid mistakes about systems with cache strategies and remote references.

Using the system notation (defs. for R, T, C, and P above) and adding:

Notation	Definition
`A@X`	"A is at X", or "the resource for A is at the X machine or at the same LAN". `@X` can be: `@C` - On Client. `@S` - On Server (or on a same high performance LAN). `@O` - On another LAN, Outside server (and outside client).
A`@X` ← B`@Y`	"information A, at X, is a received information B from Y". Attribution with communication.

Outside (or cached) server architecture.

Outside server systems (or "local systems")

R ← P@O(L@O,C@O)

The system act only on local transfer process. The "global transfer process" need two stages:

R@O ← P@O(L@O,C@O) Output production, with the system.
R@C ← R@S ← R@O Publication (using another system or something like manual FTP) and distribution (e.g. HTTP browsing).

Server-side systems there no flow between nets, all are server-side net (or server machine).

R@C ← P@S(L@S,C@S) Output "on-demand production", "on-fly publication" and receiving (over distribution method).

Or caching on server:

R@C₁ ← R_cache@S ← P@S(L@S,C@S) Output "on demand production" (first request) and caching.
R@C₂ ← R_cache@S (next request), using the cache.

Note: system generating meta-templates (like a CMS generating a PHP output) use cache also for the first request.

Client-side and distributed (decentralized) architectures.

Distributed systems All other combinations, with one or more elements, but not all, on sever:

R@S1 ← P@S2(L@S3,C@S4) Generic case.

R@C ← P@C(L@S1,C@S2) Typical case.

There are also, on distributed systems, the possibility of use a "distributed library", L, where the templates are not at the same resource.

Note: when a system with a outside server engine do also the publication, it is characterized as a distributed system:

R@S ← R@O ← P@O(L@O,C@O)

Typical server-side systems when producing with a pre-determined demand, can also use similar strategy, to "cache by publication".

Language characterization

Informally a simple template T is a "document with holes", where holes are place-holders or macro references. A template T, from the template system reference model, is an input it self, or an element from the library.

Template characterization

A template T is a string that can be split (using "hook criteria") into 2 distinct, not empty, token types:

t: output document contiguous fragments.
s: script contiguous fragments, like expressions or instructions — simple instructions, or statements, or directives, or blocks of them.
Note: a sequence of repeated s, like occurs with XSLT or ColdFusion, is transformed into a unique "contiguous s" block.

The resulted sequence of tokens is not arbitrary, and, theoretically the "contiguous hypothesis" enforce a pattern that avoid validation necessity. Technically it is validated by a regular expression: /^((t(st)*s?)|(s(ts)*t?))$/.

Formally it is supplied by a generative grammar, $G=(N,\Sigma ,Q,A)$ , with $N=\{A,X,Y\}$ , $\Sigma =\{t,s\}$ , $A$ the start symbol, and $Q$ the following production rules: $A:=X|Y|s|t$ ; $X:=tsX|ts|tst$ ; $Y:=stY|st|sts$ .

Notes:

About convention for "embed" terminology: if the template T is generated by

X

productions (starts with t), it is a template with "output language embedded with the script", else (starts with s) it is a template with a "script embedded with the output language". Languages like XQuery permits both of the "template embeddeding modes".

About point of view: designers see the script fragments as "holes", then, designers always see (by a background effect or viewer/editor choose) a template as a "output language embedded with a script".

About Parr definition: this definition is given by a generalization over "Parr split model" ^[2], that must start with t and not is submitted to system context considerations.

... Somente citar Parr e outros, sem se aprofundar, remetendo a trabalhos futuros...

References

^ ""Dual Syntax for XML Languages", C. Brabrand, A. Moller, M. I. Schwartzbach. (2006). University of Aarhus, Denmark.
^ "Enforcing Strict Model-View Separation in Template Engines", T. Parr (2004). In Proceedings International WWW Conference, New York, USA. Access also at USFCA University.

[Brad06-1] ""Dual Syntax for XML Languages", C. Brabrand, A. Moller, M. I. Schwartzbach. (2006). University of Aarhus, Denmark.

[Parr-2] "Enforcing Strict Model-View Separation in Template Engines", T. Parr (2004). In Proceedings International WWW Conference, New York, USA. Access also at USFCA University.

[1]

[2]