20 Years of Websites Evolution: Part 1 – Content Publishing


This is the part 1 in the series: 20 Years of Websites Evolution.

In the series introduction we described evolution as a struggle, and we can find no better example of that struggle than in the story of Content Publishing.

Content publishing is the most important requirement of a site – the ability to present content to visitors. Sites began as a marketing tool – a place for companies to publish their offering for the whole world (or at least those who could access the internet in those times). But in order to do so they had to overcome one big limitation – building websites required learning the technology which they were built upon – HTML (Hypertext Markup Language). Learning how to create websites using HTML can be very hard, especially for a non-programmer, which is why companies had to find professionals who could create those sites. This need had caused the rise of a new profession – programmers who specialize in the web – which are called web developers.

With a dedicated web developer companies could design, develop and publish a site. But the interaction between companies and web developers was not easy – in fact it was a constant point of strife over the last 2 decades. Trying to find the right balance between conflicting ideas like design vs flexibility, structured vs unstructured data, content vs presentation, and power vs responsibility resulted in a long tug-of-war struggle between the two sides, which had not concluded even to this day.

We can roughly divide the conflict into 5 different phases:

  • Phase 1: Static Sites (the “one man show” phase)
  • Phase 2: The rise and fall of Visual HTML editors
  • Phase 3: Dynamic Sites – the beginning
  • Phase 4: Custom back-office & Web frameworks
  • Phase 5: The age of Content Management Systems (CMS)

In this part we will learn how the conflict started and how it helped to shape the way websites are being developed today.


Phase 1: Static Sites (the “one man show” phase)

Web sites started mainly as a way for an organization to publish its content and make it accessible to the world. In the early 2000, static HTML pages (HTML is the language of web browsers) with some images were enough for that purpose. Creating and maintaining those page was the job of a one person – the all mighty Webmaster.

Webmasters created the pages using simple text editors like Microsoft Notepad and Vim, or more specialized HTML editors like Bluefish, Sublime and Coda which included features like Syntax highlighting and Autocomplete.

Once ready, they uploaded the pages and their resources (CSS/Images) using FTP/SSH/SFTP/FTPS software like WS_FTP, FileZilla or by using WebDAV clients.

Static sites were a good start but very quickly companies encountered – they were completely depended on the site webmaster. This caused some serious business problems:

  • Smaller companies needed to pay the webmaster every time they wanted to update their site content. This was not cheap and they had to wait for their webmaster availability (they were usually occupied with other customer sites).
  • Larger companies needed hire full-time webmasters and they were still depended on him – if they wanted to update the site during weekends or nights they couldn’t, and if he was sick or went on vacation they were left without the ability to bring new content to the site.
  • This model was also not much fun for the webmasters themselves, since updating content wasn’t something they really wanted to do (they wanted to work on the site code, not its content).

In the end the clients felt that even though they have paid for the site, it wasn’t truly theirs. They felt like they didn’t have control.


Phase 2: The rise and fall of Visual HTML editors

The clients wanted to bring the control back to their hand and to be able to create and update pages without being depended on the site webmaster. But they had a problem – they did not know how to HTML or CSS, which are the basic languages of the web. They needed a way to write webpages using their exiting knowledge set (which was mostly composed of using Office software).

This need brought the rise of the unholy line of tools called Visual HTML editors (aka Visual Web Design tools), which promised non-technical users that they could build an entire page without ever learning HTML or CSS. In their core lies the dreadful WYSIWYG editor.
WYSIWYG (an acronym for “what you see is what you get”) purpose was to allow the creation of HTML text by using a Word like interface (which is more familiar for most people). It worked by allowing users to edit the page like they would normally edit a word document (including the ribbons and buttons), and when they save the WYSIWYG would generate the HTML markup which corresponded to their design.

And this is how with the help of WYSIWYG editors (like Adobe Dreamweaver and Microsoft FrontPage), clients created and uploaded webpages directly to the site, completely eliminating their need for the site developer.

Or at least that was the theory. But it did not took a long time before the disadvantages of using those tools were starting to show up:

  • Visual HTML editors were difficult to use and required many months of training before mastering them. Users usually had to buy the official guide or get someone to help them learn
  • Visual HTML editors like Adobe Dreamweaver were very expensive and paying to learn them also added to the costs
  • They produced a large amount of needless code which made the page load slow and not maintainable (even not by professional web developers)
  • The end result did not look good because of the messy markup and lack of fine control web designers usually have
  • They had no template support which meant every page had to be updated by its own
  • Also no separation of design and content – meaning changing the site look required manually changing many pages
  • No built-in support for mobile devices or accessibility
  • It was hard to create dynamic pages – adding user interactivity to the page requires JavaScript, and it was very difficult to define page functionality using those tools and without knowing it
  • More reasons described here and here

In fact the base promise of WYSIWYG tools, which is that non-technical users could create web pages without learning how they actually worked, has failed. In the end since those tools couldn’t create high quality and maintainable sites, customers realized that Visual HTML editors were just not a good replacement for web developers and so they were abandoned.

Note: Some HTML editors like Adobe Dreamweaver have improved significantly over time and support many new features, but they are still usually too complicated to be used by a non-technical client, instead they are the work tools of professionals web designers working for web agencies. They are also only used for small sites with few pages only.


Phase 3: Dynamic Sites – the beginning

Since Visual HTML editors failed to deliver on their promise, a new solution was required – one which would allow the client to change site data while still leaving the control of the site code to the site developers. And in order to do so Dynamic Pages were needed.

The idea of a dynamic site was simple: instead of just returning a fixed page – the server would execute code which would change the page before it is returned, usually by adding data from a database. This technique enabled site content to be updated separately than the actual site markup, which alleviated some of the webmaster dependency.

In order to return the “processed” page, new server-side scripting languages were created like CGI, PHP, JSP, ColdFusion, and ASP. Using those languages webmasters were able to “bind” page content to a database (usually by using SQL) , which allowed having one page that return different content (usually based on the Query String part of the URL).
Server side scripting languages needed special kind of software to run which is called Web Server – for example Microsoft IIS, Apache HTTP Server or Nginx. Understanding how web servers work is required to be able to operate such sites.

But we are still missing one piece of the puzzle – which is how to allow clients to manage the data inside the database.

While site developers have no problem entering the database management system and modify the data, that was clearly not something a content editor could do (DB systems are complicated and are only meant to be used by IT professionals). Another alternative was needed, and the first and most simple solution was to make them to use software which they already knew – like Microsoft Excel.
The idea was simple: developers would map the existing page fields to excel spreadsheet columns, and allow editors to modify the excel with the desired site content. An automated process will then transfer the data into the site database.

The first attempt for such automated process was done with the built-in scripting language which came with excel (Visual Basic) to push the changes into the database. Later methods went the other way around, and pulled the data from the excel (developers usually had to write scripts for that). But which any direction they went, using excel as a content source had its own problems:

  • It only allowed 1 editor to manage the site content, since having more than one excel was going to create synchronization issues
  • For each little piece of content that editors wanted to appear in the page, a dedicated column in the excel was required
  • Adding new field required developer support and change to the main excel, which was risky and out of developers hand
  • Page fields were limited by the excel column types and formats, which do not always match the database types and formats
  • Excel was lacking in input validation, which meant a single space before or after the cell value, or even accidental cell formatting caused interpretation failures.
  • There was no “rich text editor” field, which meant only short sentences and no HTML formatting (there was no way for editors to control the display of their text)
  • It was hard to sync the excel spreadsheet with the database table because editors could add, remove and change rows as they saw fit and it was very difficult for computer to understand what exactly changed from the previous update.

After a short period of time, it was obvious excel was not good content editing tool and a new solution was required. Clients requested better user interface tools so they can upload the content to the site, which led to the creation of custom back-office systems.


Continue the story with the next part – the rise of Content Management Systems.

Next part: Part 2 – Content Management Systems.

What do you think?