HTML

HTML is the language of the web. So, why not write everything in HTML? The reasons to discard this option are well known. Let's quickly recapitulate them:

• HTML is cumbersome to write. Nobody wants to write XML code by hand, although editors with HTML/XML support might help.

• Some frequent writing tasks require non-trivial HTML code.

  Suppose we want to display a horizontally centered image with a simple black border and a link. The HTML code an inexperienced user would expect to work could look like this:

  <img src="ball.png" align="center" border="yes" link="http://www.example.com/ball">

  But the code he or she actually has to write is cumbersome and there are different ways to do it. Here is one way:

  <div style="text-align: center">
      <a href="http://www.example.com/ball">
          <img src="ball.png" style="border:1px solid black;">
      </a>
  </div>

• HTML lacks "productivity features for writers", such as:

  • Automatic generation of a table of contents, index, glossary, etc.

  • Variables used to hold recurring values

  • Splitting a document into different files

• Other inconveniences will be shown later.

Markdown

Markdown is a very popular, lightweight markup languages. It is easy to learn and use, and well suited for short and simple texts, such as comments in forums, readme files, etc.

However, it suffers from the following problems that make it unsuitable for complex or big documents (e.g. technical articles, user manuals, and books):

• The original Markdown defined by John Gruber lacks many features expected by writers, such as tables (only embedded HTML tables are supported), automatic generation of table of contents, syntax highlighting, file splitting, etc.

• There is no unique, unambiguous specification for Markdown. Many flavors of Markdown exists, with different rules and different features supported. This leads to incompatibility issues when markup code is shared. CommonMark is an attempt to solve this problem. However, the specification is huge and not completed yet (at the time of writing (February 2019) version 0.28, dated 2017-08-01, is the latest one).

• Markdown has similar problems and limitations to those shown in chapter Inconveniences / Part 2. These flaws can quickly become an annoyance when you use Markdown for anything else than short, simple texts.

Here is a list of articles with more information about Markdown's shortcomings:

• Why You Shouldn’t Use “Markdown” for Documentation

• Sundown on Markdown?

• Why Markdown Is Not My Favourite Language

Docbook

Docbook is an XML-based markup language that uses semantic tags to describe documents.

It has probably the most complete set of features among all markup languages. It has been used by many authors, is pre-installed on some Linux distributions, and is supported by many organizations and publishers. Docbook has been successfully used to create, publish, and print lots of big documents of all kinds.

But it has the following drawbacks:

• It uses XML and a verbose syntax.

  Look at the following example, borrowed from Wikipedia:

  <?xml version="1.0" encoding="UTF-8"?>
  <book xml:id="simple_book" xmlns="http://docbook.org/ns/docbook" version="5.0">
      <title>Very simple book</title>
      <chapter xml:id="chapter_1">
          <title>Chapter 1</title>
          <para>Hello world!</para>
          <para>I hope that your day is proceeding <emphasis>splendidly</emphasis>!</para>
      </chapter>
      <chapter xml:id="chapter_2">
          <title>Chapter 2</title>
          <para>Hello again, world!</para>
      </chapter>
  </book>        

  Would you enjoy writing and maintaining such code?

  Now compare the above code with the following one, written in a modern markup language like Asciidoctor:

  = Very simple book
  
  == Chapter 1
  
  Hello world!
  
  I hope that your day is proceeding _splendidly_!
  
  == Chapter 2
  
  Hello again, world!

• Docbook is complex, and therefore hard to learn and use.

• Output produced by Docbook, especially HTML, looks old-fashioned (see examples on its website). Of course, presentation can be customized, but this is not an easy task.

LaTeX

LaTeX is a high-quality typesetting system. It is widely used in academia to create scientific documents. It is considered to be the best option for writing PDF documents containing lots of mathematic formulas and equations.

I never used LaTeX myself, because I don't write scientific documents - just articles and books to be published on the web. Therefore, I don't want to comment on it too much. However, it is important to mention it because of its popularity in academia.

LaTeX's unique syntax seems verbose to me, and a bit complex. Here is an abbreviated example from Wikipedia:

\documentclass{article}
\usepackage{amsmath}
\title{\LaTeX}

\begin{document}
    \maketitle
    \LaTeX{} is a document preparation system ...

    % This is a comment
    \begin{align}
        E_0 &= mc^2 \\
        E &= \frac{mc^2}{\sqrt{1-\frac{v^2}{c^2}}}
    \end{align}  
\end{document}

The article Conversion from (La)TeX to HTML states that converting LaTeX math to HTML is "a challenge".

Some markup languages allow LaTeX snippets to be embedded in their markup code, which can be very useful if you need the power of LaTeX for maths. There are other options to display maths on the web, such as Mathjax or MathML (an ISO standard and part of HTML5).

Popular for Big Documents

A impressive number of markup languages have emerged. Many of them use a syntax similar to Markup, and are therefore easy to learn and use. Some have more features than Markdown and are even extensible. However, as soon as we start writing complex documents, corner-cases and limits diminish the initial joy of using them.

Two popular markup languages used for big documents are Asciidoctor (an improved version of Asciidoc), and reStructuredText (an improved version of StructuredText). We will have a look at them soon.

Font Styles

Font styles (italic, bold, monospace, etc.) are often used in all kinds of documents, so good support is essential.

But as we will see, surprises and limits can emerge, as soon as we have to deal with non-trivial cases. Let's look at some examples to illustrate this.

Nested Chapters

Suppose we are writing an article titled "New Awesome Product" that contains four chapters. The structure looks as follows:

New Awesome Product
    Introduction
    More features
    Faster
    Less resources

Later on we decide to insert chapter "Advantages" as a parent of the three last chapters. The structure now becomes:

New Awesome Product
    Introduction
    Advantages
        More features
        Faster
        Less resources

What are the changes required in the markup code to pass from version 1 to version 2? Can you simply insert the code for a new chapter? Let's see.

<h1>New Awesome Product</h1>
<h2>Introduction</h2>
<h2>More features</h2>
<h2>Faster</h2>
<h2>Less resources</h2>

<h1>New Awesome Product</h1>
<h2>Introduction</h2>
<h2>Advantages</h2>
<h3>More features</h3>
<h3>Faster</h3>
<h3>Less resources</h3>

= New Awesome Product

== Introduction

== More features

== Faster

== Less resources

= New Awesome Product

== Introduction

== Advantages

=== More features

=== Faster

=== Less resources

*******************
New Awesome Product
*******************
Introduction
============
More features
=============
Faster
======
Less resources
==============

*******************
New Awesome Product
*******************
Introduction
============
Advantages
==========
More features
-------------
Faster
------
Less resources
--------------

{doc title=New Awesome Product
    {ch title=Introduction}
    {ch title=More features}
    {ch title=Faster}
    {ch title=Less resources}
}

{doc title=New Awesome Product
    {ch title=Introduction}
    {ch title=Advantages
        {ch title=More features}
        {ch title=Faster}
        {ch title=Less resources}
    }
}

Lists

In Asciidoctor the kind of problems we have seen with chapter hierarchies can also arise with list hierarchies (e.g. lists that contain lists). The reason is the same as for chapters: Asciidoctor lists use different markup code to explicitly specify the level of list items (* for level 1, ** for level 2, etc.). Moreover, there are a number of complications you have to be aware of when working with complex list content.

In reStructuredText, nested lists are created using indentation and blank lines. This works fine for simple nested lists, but creates other problems in more complex cases (not discussed here). Using whitespace (e.g. blank lines and indentation) to define structure in markup code is a bad idea, as we'll see later.

In HTML and PML, the above problems don't exist with lists because the syntax for parent- and child nodes is the same (not shown here).

Comments

Good support for comments is important, because they are used frequently, especially in collaborative environments. So, how well are they supported?

{-
These lines
will not {- nested comment -}
appear in the final document.
-}

Whitespace And Indentation

At first, using whitespace to define structure seams like a good idea. Look at the following example:

parent
    child 1
    child 2

The structure is very easy to read and write. No noisy special markup characters are needed.

It is therefore tempting for markup language designers to use whitespace as a simple way to define structure. Unfortunately, this works well only for simple structures, and has other inconveniences we'll see soon.

Therefore, a simple, but important rule must be applied in markup languages designed to work well with complex content:

Whitespace doesn't change the structure or semantics of the document.

whitespace-insignificant-rule

What does this mean?

First, let us define whitespace: Whitespace is any set of one or more consecutive spaces, tabs, new lines, and other Unicode characters that represent space.

In our context, the above rule means that:

• Within text, a set of several (i.e. more than one) whitespace characters is treated the same as a single space character.

  For example, this code:

  a beautiful
      flower

  ... is identical to this one:

  a beautiful flower

• Between structural elements, a set of whitespace characters is insignificant.

  For example, this code:

  <table>
  
      <tr>

  ... is identical to this one:

  <table><tr>

A special case of whitespace is indentation (leading whitespace at the beginning of a line). The above rule implies that indentation is insignificant too. Indentation doesn't change the result of the final document.

Applying the whitespace-insignificant rule is important, because it leads to significant advantages:

• There is no need to learn, apply and worry about complex whitespace rules (see examples below).

  Violating the whitespace-insignificant rule in a markup specification adds unneeded complexity, and can lead to markup code that is ugly, error-prone, and difficult to maintain, especially in the case of nested lists.

• Whitespace can freely be used by authors to format the markup code in a more understandable, presentable and attractive way (pretty printing). For example, lists (and lists of lists) can be indented to display their structure in a visually clear and maintainable way, without the risk of changing the underlying structure.

• Text blocks can be copy/pasted without the need to adapt whitespace.

• If shared text blocks (stored in different files) are imported into several documents with different structures, there is no risk of changing or breaking the structure.

• There is no unexpected or obscure behavior if the whitespace is not visible for human readers. Some examples:

  • a mixture of whitespace characters, such as spaces and tabs, especially when used to indent code

  • whitespace at the end of a line

  • whitespace in empty lines

  • visually impaired (blind) people who can't read whitespace

  Note: To alleviate the pain, some editors provide a display-whitespace mode.

• Tools that generate markup code, as well as markup parsers are generally easier to create.

• In some situations it is useful to reduce whitespace to a minimum (e.g. no new lines), in order to save storage space and improve performance.

If you want a few examples demonstrating the kind of technical problems that arise if whitespace is significant, you can read:

• What are the downsides to whitespace indentation rather than requiring curly braces?

• F# syntax: indentation and verbosity

• Issue in nodeca/js-yaml

So, how is whitespace handled in the languages we are discussing in this article?

• HTML

  HTML applies the whitespace-insignificant rule.

  For a thorough explanation, look at this excellent article written by Patrick Brosset: When does white space matter in HTML?.

• Asciidoctor

  In Asciidoctor, whitespace is significant in some cases.

  This can lead to surprising behavior and problems with no easy or no satisfying solution. Some examples can be seen here and here.

• reStructuredText

  reStructuredText has whitespace rules that are 'a bit surprising'.

  For example, writing *very* results in very (text in italics, as expected). However, * very* results in * very* (no italics!), because of the whitespace preceding "very". To understand why, the answer might be found in chapter Whitespace of the specification.

• PML

  Similar to HTML, PML applies the whitespace-insignificant rule.

  There is one exception: For practical reasons, a blank line between two text blocks results in a paragraph break. This means that instead of writing:

  {p text of paragraph 1}
  {p text of paragraph 2}

  ... we can simply write:

  text of paragraph 1
  
  text of paragraph 2

Other Inconveniences

As seen already, some markup languages systematically use opening and closing tags. An example would be <i> and </i> in HTML. All XML-based languages, as well as PML belong to this class of languages.

Without digging into details, here are some drawbacks that can occur in languages that do not (or not always) use pairs of distinct opening/closing tags (e.g. Markdown, Asciidoctor, and reStructuredText):

• Editor support

  Creating good, reliable editor support is more difficult to develop. Examples of useful editor features are:

  • syntax highlighting for markup code

  • markup code completion

  • visualizing pairs of block start/end marks (e.g. { and its corresponding })

  • block collapsing/expanding

  In the case of languages that use distinct opening/closing tags, the two last features work out-of-the-box in some editors. For example, PML uses { and } for node boundaries. This is also used in many programming languages (C, Java, Javascript, etc.) and therefore block features implemented for programming languages will also work for PML.

• Document validation

  Fewer syntax and structure errors can be detected automatically. This can lead to more time spent on debugging documents. Or, even worse, there might be silently ignored errors that end up in wrong output (Did I really fail to spot the missing warning block on page 267 of my 310 pages book?).

• Parsers

  It is more difficult to create parsers (i.e. programs that read markup code) that work well in all cases. If different parsers read the same markup code, there is an increased risk of getting different results for corner-cases.

• Auto-generated markup code

  Tools that generate markup code programmatically are more difficult to create. For example, if whitespace is significant, or font styles cannot be nested, then additional state must be updated and tracked, in order to respect these rules.