Add blog post source (not 100% up to date with text on blog though).

Author: hlindberg

about-scope.md

@@ -0,0 +1,64 @@
+Scope Stuffs
+===
+
+Node Scope
+---
+* People want access to node scope
+* What is node scope vs. top scope
+* What happens with shadowing when loaded code sets things in top scope
+
+Access to Node Scope
+---
+For node scope to become addressable it needs to be in a namespace, and this namespace
+is always the same (since users will reference things in "current node scope" (not for some
+other node).
+
+Alternatively, this can be available via a type. Node[name] could be a reference to a particular
+node's scope. ThisNode could be a reference to the current evaluating node. The later could accept reference to its parameters - i.e. `ThisNode[msg]` would get $msg in nodescope.
+
+We can also create a named scope, even setting of a variable in node scope (say $x) becomes
+a setting in `$nodescope::x`, and this thus both shadowing and globally accessible.
+
+Node Scope vs Top Scope
+---
+
+If autoloaded code has code in top scope - what should happen when a variable is set?
+
+* It is set in top scope and potentially (already) shadowed in node scope
+* It is set in node scope, and potentially clashes (mutation attempt error)
+* node scope does not exist - a new top scope being a merge is set as the global scope (and 
+  all top scope settings after that are clashes
+  
+We may also forbid all top scope assignments in auto loaded code.
+
+Match Scope
+---
+A match scope is constructed based on a regular expression match. There is however a problem
+in that a sequence of statements should retain the last match scope. For this reason,
+scope must handle match result in a special way.
+
+When a match is made, the match data should simply be set in scope. When entering an expression
+that preserves the current match data, it should create a new MatchScope (transparent to everything
+except match data).
+
+Performance Improvements
+---
+Numeric variable detection is a performance problem, every lookup in every scope has to check
+if the variable name is numeric. We can solve this by creating a special instruction
+for numeric variables and have a special lookup of these via dedicated lookup / exists. This
+then completely removes the need to look at the variable name string.
+
+We can probably also speed up by differentiating between SimpleName, and QualifiedName. Thi
+because we only have to check for '::' when it is constructed. We may also keep the value
+around in both concatenated and split form to avoid having to split and combine it multiple
+times.
+
+Discussion
+===
+
+Node Scope is a transaction specific context around the logic being evaluated. If it shadows
+there should really be no way to break out from that shadowing and read the non shadowed variables.
+(i.e. no breaking of the encapsulation).
+
+This means that all top scope assignments (if allowed) in autoloaded code should be set in
+the node scope - i.e. there is no difference between it and the top scope and they could just as well be merged right away.

blog-posts/heredoc.md

@@ -0,0 +1,132 @@
+Heredoc is here! Starting with Puppet 3.5.0 with --parser future turned on you can now
+use Puppet Heredoc; basically a way to write strings of text without having to escape/quote special
+characters. The primary motivation for adding heredoc support to the Puppet Programming Language
+is to help avoiding the problem known as "backslash hell", where every backslash character in a
+string may require, two, four or more backslashes to pass an actual backslash through multiple layers
+of string special character interpretation.
+
+Before talking about the features of Puppet Heredoc, lets look at an example:
+
+     $a = @(END)
+     This is the text that gets assigned to $a.
+     And this too.
+     END
+
+As you probably already figured out, the `@()` is the heredoc start tag, where you get to
+define what the end tag is; a text string that marks where the verbatim sequence of text on the lines
+following the start tag ends. In the example above, the end tags is `END`. (Obviously you have
+to select an end tag that does not appear as a separate line inside the actual text).
+
+This blog post is a brief introduction of the Puppet Heredoc features, the full specification
+is found in the [Puppet Heredoc ARM-4 text][1].
+
+[1]:https://github.com/puppetlabs/armatures/blob/master/arm-4.heredoc/heredoc.md
+
+### Trying out the examples
+
+If you want to try out the examples, you need Puppet 3.5.0 and then turn on --parser future.
+
+### Controlling the Left Margin
+
+A problem with Heredoc is how to deal with text that appears in indented text, but you
+do not want the indentation in the resulting string.
+
+    if $something {
+      $a = @(END)
+      Text here is indented 2 spaces.
+      END
+    }
+    
+Puppet Heredoc solves this by allowing you to define where the left margin is by using
+a pipe `|` character on the end-tag line at the position where the first character on each line
+should be. To fix the example above, we then write:
+
+    if $something {
+      $a = @(END)
+      Text here is indented 2 spaces.
+      | END
+    }
+
+### Controlling trailing new-line
+
+Another problem with heredoc text is how to deal with the line ending of the last line
+of text (and any trailing whitespace on that line). With Puppet Heredoc you can easily strip
+out trailing space and the newline by using a `-` before the end tag. (You can combine the - with | by placing the - after the pipe).
+
+Here is the same example again, now without trailing new-line in the result:
+
+    if $something {
+      $a = @(END)
+      Text here is indented 2 spaces.
+      |- END
+    }
+
+### Interpolating variables
+
+The default mode of Puppet Heredoc is to not interpolate variables (e.g. having `$a` in the
+heredoc text does not expand to the value of the variable `$a`). If you need this, it is possible
+to turn on interpolation by double quoting the specification of the end tag.
+
+    $a = world
+    notice @("END")
+    The $a is an awesome place
+    |- END
+
+Will output "The world is an awesome place".
+
+Naturally, since there also needs to be a way to enter a `$`, escaping is turned on for `$` and for `\`. Thus when using interpolation, a `\` must be entered as `\\`, and a `$` as `\$`.
+
+You can use both styles of interpolation; either just `$a`, or `${a}`. The same rules for interpolation of expression as for double quoted strings apply.
+
+### Controlling Special Character Escapes
+
+By default, all character escapes are turned off (when using interpolation escapes for \ and $ are turned on). Puppet Heredoc also allows you to control escapes in more detail. The possible escapes are t, s, r, n, u, L, and $, and you can control these individually by specifying them in the heredoc start tag like this:
+
+    $a = @(END/tL)
+      This text has a tab\t and joins this line \L
+      with this line.
+      |-END
+
+Most of the escapes should be familiar, except the `L` escape which makes it possible to escape
+the end of line thus effectively joining a line with the next. The charters may appear in any order
+in the spec. Using one (or more) escapes also always turn on escaping of `\`.
+
+### Specifying the Syntax
+
+The Puppet Heredoc start tag allows specification of the syntax of the contained text. This is done by following the end tag name with ':', and the syntax/language specification as a mime specification string following the ':'. Here is an example:
+
+    $a = @(END:json)
+    ["a"]
+    - END 
+
+The syntax/language tag serves dual purpose; it is an indicator to tools (such as Geppetto) how the tool should perform things like syntax highlighting or syntax checking, and it enables the Puppet Parser to perform syntax checking if there is a plugin that checks the given syntax. 
+
+In Puppet 3.5.0, there is a syntax checker for Json, and consequently, if you were to enter the
+following example, you will see it report the Json syntax error.
+
+    $a = @(END:json)
+    ['a']
+    - END 
+
+You will get the following error:
+
+    Error: Invalid produced text having syntax: 'json'. JSON syntax checker: Cannot parse invalid JSON string. "unexpected token in array at ''a']'!"
+
+New syntax checkers can be written in Ruby, and distributed as a Puppet Module. (This will be the topic of a future blog post).
+
+### Summary
+
+This blog post is an introduction to Puppet Heredoc. There are some additional features that are documented in the full ARM text, such as how to use multiple heredocs on the same line, the precise
+semantics of special character escapes and margin control, the details about what is permissible as
+an end-tag etc.
+
+What better then to end with some poetry...
+
+    notice @(Verse 8 of The Raven)
+      Then this ebony bird beguiling my sad fancy into smiling,
+      By the grave and stern decorum of the countenance it wore,
+      `Though thy crest be shorn and shaven, thou,' I said, `art sure no craven.
+      Ghastly grim and ancient raven wandering from the nightly shore -
+      Tell me what thy lordly name is on the Night's Plutonian shore!'
+      Quoth the raven, `Nevermore.'
+      | Verse 8 of The Raven

blog-posts/internals/modeling/modeling-intro.md

@@ -0,0 +1,157 @@
+Puppet is a model driven system - a desired system state is modeled by describing the
+desired state of a set of resources, these are compiled into a catalog, and the catalog is
+applied to a system to make it have the desired modeled state.
+
+What is new in the "future parser/evaluator" features in Puppet is that we started using modeling technology to implement how puppet itself works.
+
+In this post, I am going to talk a bit about modeling in general, the Ecore modeling technology
+and the RGen Ruby implementation of Ecore.
+
+### What is a "model" really?
+
+If at first when you hear the word *model*, you think about Tyra Banks or Marcus Schenkenberg and then laugh a little because that is obviously not the kind of models we are talking about here, you are actually both right and wrong at the same time.
+
+A model is an abstraction. We use abstractions all the time; when we talk about something like a "Car" we are talking about an unspecific instance of something like "a powered means of transportation, probably a 4 door sedan". Fashion models such as Tyra or Marcus are also abstractions of "people wearing clothes" albeit very good looking ones - in a way like if we design a Car icon to depict a Lamborghini, but now I am beginning to get off topic and into a completely different debate.
+
+We can express a model concretely:
+
+    A Car has an engine, 2 to 5 doors, a steering wheel, breaks, and 4 wheels.
+    . . .
+    
+We can also express such a model in a programming language:
+
+    class Car
+      attr_accessor :engine
+      attr_accessor :doors
+      attr_accessor :steering_wheel
+      attr_accessor :breaks
+      attr_accessor :wheels
+      
+    . . .
+    end
+
+As you can see in the above attempt to model a Car we lack the semantics in Ruby to declare
+more details about the Car's attributes - there is no way to declaratively state 
+how many doors there could be, the number of wheels etc. There is also no way to declare
+that the engine attribute must be of engine type, etc. All such details must be implemented 
+as logic in the setter and getter methods that manipulate a Car instance. While this is fine
+from a runtime perspective (it protects us from mistakes when using the class), we can not
+(at least not easily) introspect the class and deduce the number of allowed doors, or the allowed type of the various attributes.
+
+While Ruby is a very fluid implementation language, in itself it is not very good at
+expressing a model.
+
+### Modeling Language
+
+A modeling language (in contrast to an implementation language such as Ruby) lets us
+describe far more details about the abstraction without having to express them in imperative
+code. One such family of "languages" are those that are used to describe data formats - they
+are referred to a schemas - and you are probably familiar with XmlSchema, JsonSchema, Yamlschema etc
+that allows a declaration to be made about what is allowed in data that conforms to the schema.
+
+A schema is a form of modeling language. What is interesting about them is that it now possible
+to transform between them! Given an XmlSchema, it is possible to transform it into a corresponding
+Yaml or JsonSchema, and likewise transform data conformant with one such schema into data comformant
+with the transformed schema. (The problems doing this in practice has to do with the difference in semantic power between the different technologies - we may be able to express rules/constraints
+in one such schema technology that does not exist the others).
+
+### Schemas and Meta Models
+
+When we look at a schema - we are actually looking at a meta model; a model that describes a model.
+That is if we describe a car in Json we have a car model:
+
+    { "engine": "combustion",
+      "steering-wheel": "sport-leather",
+      "wheels": ...
+    }
+    
+And if we describe a schema for it:
+
+    { "title": "Car Schema",
+      "type"; "object",
+      "properties": {
+        "engine": { "type": "string"},
+        "steering-wheel": { "type": "string" },
+        . . .
+      }
+      "required": ["engine", "steering-wheel", ...]
+    }
+    
+We have a Car meta-model.
+
+In everyday speak, we typically refer to the schema as "schema" or "model" and simply ignore
+its "meta status". But since we are on the topic of meta models - what we can do now is to
+also express the meta model as a model - i.e. what is the schema for a jsonschema?
+Here is an excerpt from the [Json "Core/Validation Meta-Schema"][1]
+
+    {
+      "id": "http://json-schema.org/draft-04/schema#",
+      "$schema": "http://json-schema.org/draft-04/schema#",
+      . . .
+       "title": {
+            "type": "string"
+        },
+      . . .
+      "required": { "$ref": "#/definitions/stringArray" },
+      . . .
+    }
+
+If you are interested in what it looks like, do download it. Be warned that you will quickly become
+somewhat disoriented since it is a schema describing a schema that describes what Json data
+should look like...
+
+[1]:http://json-schema.org/documentation.html
+
+A meta schema such as that for jsonschema is very useful as it can be used to validate schemas
+that describe data.
+
+Schemas such as these are good for describing data, but they becomes somewhat difficult to use
+in practice for the construction of software. There are other modeling languages that are more specifically targeting software system constructs. There are both graphical and textual languages
+as well as those that have both types of representations.
+
+What we are specifically interested in is an [Object modeling language][2]
+
+[2]:http://en.wikipedia.org/wiki/Object_modeling_language
+
+### Levels of reality / meta-ness
+
+| level | Description |
+| --- | --- |
+| M0 | Real Object - e.g. the movie Casablance on a DVD |
+| M1 | User Model / Instance Level - e.g. a computer abstraction of the DVD - `aVideo = Video.new('Casablanca')` |
+| M2 | Meta Model - e.g. defines what `Video` is, its attributes and operations |
+| M3 | Meta meta model - e.g. defines how the definition of what a `Video` is, is expressed |
+
+
+### Object Modeling Language
+
+An Object Modeling Language is a language that directly supports the kinds of elements we are interested in when constructing software. E.g. classes, methods, the properties of objects, etc. You probably heard of one such technology called [Unified Modeling Language][3] - this is a broad modeling technology and is associated with object oriented software development methodologies such as Booch, OMT, Objectory, IBM's RUP, and Dynamic Systems Development Method. This was the big thing in the 90's, but UML has since then more or less slid into darkness as "the way to write better
+software". An interesting debate from 2009, can be found [here][4].
+
+[3]:http://en.wikipedia.org/wiki/Unified_Modeling_Language
+[4]:http://codebetter.com/jeremymiller/2009/09/12/how-relevant-is-uml-modeling-today/
+
+There is however a very useful part of the UML technology that is often overlooked - the so
+called Meta Object Facility (MOF) that sits at the very core of UML and it contains the meta model
+that UML itself is defined in. MOF plays the same role for models as what Extended Backus Naur Form (EBNF) plays for programming languages - it defines the grammar. Thus MOF can be said to be a DSL used to define meta models. The technology used in MOF is called Ecore - it is the reference implementation of MOF.
+
+### Ecore
+
+Ecore is part of [Eclipse EMF][5], and is heavily used within Eclipse for a wide variety of
+IDE applications and application development domains. In the Puppet Domain Ecore technology is
+used in the [Puppetlabs Geppetto IDE][7] tool in combination with additional frameworks for language development [Xtext][6]
+
+[5]:http://en.wikipedia.org/wiki/Eclipse_Modeling_Framework
+[6]:http://en.wikipedia.org/wiki/Xtext
+[7]:http://puppetlabs.github.io/geppetto/
+[8]:https://github.com/mthiede/rgen
+[9]:https://code.google.com/p/emf4cpp/
+
+Eclipse EMF is a Java centric implementation of Ecore. There are also implementations for
+Ruby ([RGen][8], and C++ [EMF4CPP][9]).
+
+Thus, there are many different ways to express an Ecore model. The UML MOF has defined one
+serialization format known as XMI based on XML, but there are many other concrete formats such
+as Xcore (a DSL built with Xtext, annotated Java, Json, binary serialization formats, the Rgen score
+DSL in Ruby, etc.) 
+