Phase 4: Custom back-office & Web frameworks
Although Excel was not a very successful content editing tool, it did had one characteristics which had potential – the use of structured data. Users had predefined fields to enter the page content, and since those fields only contains the actual text that needed to display, developers could easily wrap the field content inside the appropriate HTML markup, ensuring it will be displayed correctly in all the browsers.
This is the opposite of what happened in the previous phase, were the client output from the Visual HTML Editors tools contained both the page content and it’s design (HTML markup), which meant from the web developers standpoint it was unstructured blob of text with little control of (for example they couldn’t adjust the HTML markup to support different browsers).
Working with structured data (like excel columns or form fields) follows the principle of content and presentation separation, which helps both the client and website developers to produce high quality sites. But as we saw excel was not good enough, which let many clients to ask the website developers to create for them some kind a form which would allow them to enter the data in a more controlled and safe way. And this was start of the custom back-office age.
But before developers could start creating those custom back-office pages, another evolution was required.
Building custom back-office is a lot more complicated then a basic publishing site. Most web developers up until that time only knew how to output site markup, and not how to handle data input. And we can’t have random people using the form – it needed to be protected so only the client could use it, which required creating a membership systems (which was already starting to be required for customers zone). Their only tool they had been a basic scripting languages with manually written SQL statements, which was clearly not enough.
In order to help developers to deal with all the new requirements and the complexity which comes with them, new tools evolved. They were called Web framework, and those web frameworks helped developers solve many issues dynamic scripting languages had:
- Difficulty to make cross site changes: There are many parts of the site which are the same in all pages like the header, the menu and the footer. What happens when a site face-lift is needed and cross-site changes are to be made? If all we have are separated pages than the site developer needs to manfully make the same changes in all of the pages (could be hundreds). This is why Web Framework provide Template Processor, which allows real-time combination of different pages into one. This enables the developer make the change only once and its automatically reflected in every page in the site.
Difficulty to maintain Spaghetti code: Having all of the site code inside a single page led to something called “Spaghetti Code”, where everything was so mixed together that even the developer who wrote the code couldn’t understand it couple of weeks later. This meant the every little change took a long time and required lots of testing.
Difficult to reuse functionality: Another problem with spaghetti code was that moving functionality from one page to another was very difficult to do. Trying to isolate and separate the specific code lines took hours or days and made the customer angry (after all, if its working in one page why does it take so long to make it work in another one?) . Web Frameworks solved this issue by allowing developers to encapsulate code into separate components, which could be easily move to another page (Web template system).
Data retrieval is coupled with presentation: Since dynamic pages had to pull data from the database, they usually contained the SQL query inside of them. But this caused many issues when the query had to be changed. In many times the same query was hard-coded into multiple pages, which forced the developers to try to locate the query and fix it many times. And if he missed a page or two – only customer would later complain that the data is wrong. This is the main reason why separation between data access layer and the presentation layer was needed, and using a Web Frameworks made such separation much easier.
SQL Injection: Another related issue is that by having hard-coded SQL inside the page created a critical security vulnerability called “SQL Injection”, which is the most exploited security vulnerability in the world (still #1 in OWSAP). Web Framework usually comes with ORM (like Entity Framework) which protects against these kind of attacks.
No cache: When sites began to handle larger traffic they soon encountered a problem – they began to slow down as more people were using them. The cause was all the page processing and data retrieving that needed to occur for each page render. This is why Caching solution was required, and although some external solutions were sometime used (like Nginx or Varnish), the caching that web framework provided was better because it was integrated into the framework which allowed automatic expiration and more granular control.
No Users & Session management: The need for session management arrived when sites began offering “membership” area where they could login to get extra benefits (usually they had to pay in order to get into the members only content). Since HTTP is stateless, servers had to maintain session for each logged-in user. This is another functionality which Web frameworks provided without the need for developers to create such systems. Membership systems were also used for back-office access.
Difficulty to work in teams: As sites became richer in functionality one developer was not enough, and it was now required for two or more developers to be working at the same site. But if all the code sits in one huge hard to understand file, it was not possible for two developers to be working on the same file at the same time (especially when it is Sourced-controlled). Web frameworks allowed code separation into different files.
Difficulty to create big, complex site: The result of all of those issues meant they it was very hard to create and maintain big, complex web sited which required large teams, therefore limiting the size of sites which could be created.
Web Framework were created in order to help developers deal with all of those issues and prevent them from reinventing the wheel for every site. They also made the creation of back-office sites a lot more easier, with the help of membership system, custom forms and the built-in ORM with input validation.
Almost every server-side scripting language has one (or more) web frameworks. For example here is a list of known web frameworks: ASP.NET, Java EE, WebObjects, web2py, Ruby on Rails, Django, Zend Framework, and more.
Since web framework are lot more complex than basic server-side scripting languages, developing for them required more advanced tools than just a notepad.
This is why developers needed to learn how to use IDE (Integrated Development Environment) like: Visual Studio, Webstorm, Eclipse, Netbeans, Zend Development Environment and more. In addition of learning how to use an IDE, learning web framework also takes considerable time (usually 2-3 years).
Phase 5: The age of Content Management Systems (CMS)
Web frameworks helped developers building complex sites for both the customers (front) and the client (back-office). But they still took a lot of effort and some features were very hard to implement. The basic back-office system was based on a simple web form with fields corresponding to the fields on the page. This had some major disadvantages:
- Adding new field to a page required development which took time and money
- With only simple text fields they had to split large articles into many fields (see prev point)
- Clients didn’t have any control on how the text will be displayed (even making a word look bold was not possible)
What they needed was a way for them to define their own custom page fields and control content position, and a way to create large articles by freely mixing formatted text and images without requiring development support. And the solution for many those problems came in the form of CMS systems.
CMS are applications which are meant to create and store all the content which is used in the site itself. As such, their primary functions is to enable content editors (CMS term for clients and marketing team) to create articles by defining their own data structure. CMS came with a WYSIWYG field which is called Rich text editor(RTE) that brings back some of the Visual HTML editor capabilities, only this time limited to a single field (instead of basing the entire site design on it).
CMS also helped the developers – instead of investing most of the their time in building the back-office, they could return to their original purpose which was building websites, and not back-offices.
As time moved on CMS systems had to evolved in order to meet new requirements. Today all top used CMS are a product of more than 10 years of development, and contain many features which are needed for the creation of modern sites, for example:
- Data Structure:
- Advanced Rich text editor
- Custom Properties & Editors
- Multilingual Content (mirror sites, multilingual fields, export & import dictionaries for translation, automated translations)
- Content hierarchy, CRUD, Ordering
- Multi-level Navigation, Categories, Tags, Taxonomy
- Multi Site Management
- Dictionaries, Lookup List, Reusable Content
- Data management:
- Permissions, Roles
- Publishing Workflow, Versioning, Archive
- Notifications, Alerts
- Development Features:
- Modular design (Master, Layouts, Web Parts)
- Ability to create mobile & accessible sites
- Themes (front / back)
- Responsive media (scale & crop, set focus, thumbnails)
- Back-office Features:
- Multi-Editors support
- Members & User Management
- Live preview, Responsive preview
- Localization (for editors)
- Forms Generator
- Stability & Scalability
- Audit, Logs, Error Handling, Health Monitoring
- Cache (Data, Html, Images), CDN support
As we can see CMS offer more than simple text editing, and enable multiple editors to work on the same site without interfering. CMS are in fact the next evolution of the web frameworks – they provide many important features not only for developers but for content editors as well.
Like frameworks, there are hundreds of CMS products (each language has its own), some are paid (like Sharepoint, SiteCore) and some are free (like Drupal, WordPress).
Because of their complexity, CMS take many years to mature (the most used CMS systems today were created over a decade ago) and usually require many years to fully learn how to use them. Many developers can spent their entire career based on one CMS product, and still don’t know everything there is to know about it.
Although CMS offer the best balance between structured and unstructured content that we have, it is not always obvious when designing a new site which parts are which. Finding the right balance between those two worlds can be tricky and requires lots of experience and strong collaboration between the client, the project manager and the development team.
Content Publishing may be the most basic requirement for a website, but finding the right solution was not a simple process. It took more than 20 years and 5 attempts to get it right, and resulted with one of the most complicated products out there – Content Management Systems. But all that long struggle was not in vain – it made both sides understand each other needs, forced them to find better solutions which are suitable for both sides – in the end it helped to shape how modern CMS systems looks like.
Modern CMS systems are the result of that struggle – they are, in fact, the result of Evolution.
In the next part we will continue the story of content publishing by learning about dynamic pages and the client side evolution.
Next part: Part 3 – Dynamic Pages.