Course 7401ICT eService Technology is an intensive, software engineering course on eService Technology development. Courses 7402ICT and 7403ICT describe(d) what services to implement and why they should be implemented. Course 7401ICT focuses on how to design and implement the required services.
Requirements to succeed in the course are:
Asessment is based on three Web service development assignments (20%, 20%, 30%) and an end-of semester examination (30%). The first two assignments are done individually; the third may be done individually or in pairs.
Practical work is done on the student Linux server
dwarf.ict.griffith.edu.au. Practical work in laboratories leads
directly to the assignments.
Lectures are held on Mondays in N34 0.34 from 2pm to 4pm and laboratory classes on Mondays in N44 1.17 from 4pm to 6pm. Consultations and/or alternative classes are held at other times during the week (to be decided after discussion).
All course material is at the URL http://www.ict.griffith.edu.au/teaching/7401ICT/. It is important to note that the lectures notes here contain only a summary of the actual material presented in class.
The convenor is Prof. Rodney Topor, email r.topor@griffith.edu.au, phone 3735 5047, room N44 2.20 (preferably by appointment). Any emails addressed to Prof. Topor should be sent from your University email account and should include 7401ICT in the subject line.
The purpose of this section is to introduce you to the core standards used in implementing Web Services, to the standard architecture used for implementing such services, and to a useful design tool used for specifying them.
Beware. Much published material, particularly on the Web, is obsolete and recommends bad practice.
The World Wide Web is defined by the following three standards:
Increasingly, especially on mobile devices, applications that use TCP/IP and HTTP for communication, but do not HTML or a Web browser are called Web applications.
Also, (browser-based) Web applications increasingly use JavaScript to provide richer user interfaces than are possible with only HTML and CSS.
Before considering standards and languages in detail, it's necessary to understand the architecture of all eServices.
Client (browser) <--> Web server (plus scripting language) <--> Database server
Each layer makes requests to the following layer which sends responses back to the requester. The client/web server interface uses the HTTP protocol; the web server/database server interface (normally) uses the SQL query language.
The most important client-side software application is the Web browser. Good browsers include:
We are only interested in modern, reasonably standards-conformant browsers.
The browser manages the user interface of an eService by implementing the following three standards:
It is essential to use CSS to separate presentation from content and structure. It is optional to use JavaScript to separate behaviour from presentation.
Brief descriptions of these three standards may be found in my Web Programming lectures here (HTML and CSS) and here (JavaScript). We need to become familiar with HTML and CSS immediately; we only need tobecome familiar with JavaScript later in the course.
All three of these standards are under active development (after a long period of hiatus).
Web servers receive HTTP requests and send HTTP responses.
Each of these frameworks includes a template system to separate application logic from HTML presentation. Java EE (through Play! or Hibernate), Rails and Django all provide automatated object-relational mapping.
A brief introduction to PHP may be found in my 2503ICT lecture.
In 7401ICT eServices Technology, we shall focus on Django, as described in "the Django book" (available on paper and digitally), the "practical projects" book, the "beginning e-commerce" book, and the documentation.
Currently most eServices data is stored as either relational data or XML data (but this is slowly changing).
SQLite is a small, fast, portable, widely-used, administration-free, relational database management system, suitable for small Web applications and embedded systems. We use SQLite in this course (though you can use MYSQL if you really wish to).
A brief introduction to relational database management systems may be found in my 2503ICT lecture.
A brief introduction to the use of PHP with MySQL (or SQLite) may be found in my 2503ICT lectures here and here.
There are several open-source XML data management systems (such as eXist), and several commercial relational database systems (such as Oracle) that also manage XML data.
This section introduces two very simple Web applications that will be widely used to illustrate design and development techniquese in the course.
This first example is a simple guestbook system that illustrates how HTML, CSS, PHP, Smarty templates, and MySQL can be used together.
This second example is a simple groups management system that represents the core of Yahoo! Groups. It provides the ability to display titles of the discussion groups, to create new groups, to display headings of the messages in a group, to display the content of a message in a group, and to create a new message or reply to an existing message.
The most important feature missing from these examples is user authentication and authorisation, the requirement that users must register and login before they can perform many operations. (This is difficult with PHP, but much simpler with Django.)
These two examples illustrate the interaction of the three tiers of a Web system architecture.
Later we shall study their implementations using Django.
Discussed in 7402ICT and 7403ICT. Not covered here. Except to note that all successful, modern Web services are interactive and rely on user-provided data.
Many students find the step "Implement the design" the most challenging, but this is often because they have not done the design well and hence do not really understand what they are trying to implement.
A longer introduction to the eServices develpment process may be found in my 2503ICT lecture.
Transitions diagrams (also called interaction diagrams) are important for both specification and design. A transition diagram is a directed graph with annotations on (some) edges. The nodes of the diagram describe page views and actions (that do not correspond to page views) in the application. Nodes should be labelled by verbs, e.g., show all users, enter new user details, add new user. The edges describe transitions between nodes. The start or home node should be clearly identified.
Here is a transition diagram that specifies the page views, actions and transitions of the above guestbook system.
Here is a transition diagram that specifies the page views, actions and transitions of the above groups management system.
Transition diagrams are important for specification. Designers can use a diagram to check that all requirements have been considered and that all required transactions have been identified, and that all transitions are appropriate.
Transition diagrams are important for design. Programmers can use the annotated diagram as a guide to the actual implementation.
A single transition diagram may be implemented in different ways. With Django, each node may be annotated with the URL that corresponds to that node. With PHP, each node may be annotated with the script name that corresponds to that node. Edges may be annotated with the variables whose values are passed in the corresponding page transition. Other annotations may also be useful.
If an (HTML) template system is used, one or more separate template files will be associated with each page view in the diagram. Hence, in this case, several files will be required to implement each page view node.
XML (eXensible Markup Language) is increasingly used for storing and transferring data on the Web. Key standards and architectures include:
Simpler alternatives to XML, for restricted purposes, such as JSON, are also widely used.
A brief introduction to these standards and architectures may be found in my 2503ICT lectures here and here.
The purpose of this section is to introduce you to a selection of languages and frameworks used for Web service development in a little more detail and to briefly compare them.
See my 2503ICT lectures on HTML and CSS.
Currently, the most widely used languages and frameworks are these.
There is much evidence that in the near future, use of the following frameworks will grow rapidly.
Each of these frameworks includes a template system to separate application logic from HTML presentation.
A brief introduction to PHP may be found in my 2503ICT lecture.
The official PHP site contains a tutorial and the documentation.
See (again) the simple guestbook example and the simple groups management example.
PHP is a mature, efficient, cross-platform, open-source, well-documented, proven language that has been used for many large sites, including (much of) Yahoo! and Facebook.
Well-written PHP applications use templates to separate application logic from HTML presentation.
CakePHP, Symfony and Zend are higher-level PHP frameworks that use the MVC pattern, similar to Ruby on Rails and Django (below), but are less convenient (in my opinion). Other higher-level PHP frameworks are also emerging.
Java Enterprise Edition (Java EE) is a mature, complex platform for developing Web applications, developed and promoted by Sun. To use Java EE for Web development, it appears to be necessary to understand servlets, Java Server Pages (JSP, the level most similar to PHP), JavaServer Faces (JSF, a framework that provides templates, similar to ADO.NET), Netbeans (a framework for defining useful classes), Hibernate (an object-relational mapping library), Spring (a high-level application framework), Struts (another high-level application framework), and other technologies. You don't have to understand all these terms - I don't (yet). Play! is a new Java EE framework very similar to Django (below).
There is a steep, high learning curve required to use Java for Web application development. This is normally only undertaken for large projects in large organisations.
.NET is Microsoft's collection of technologies for Web development. At one level, using .NET is like using PHP or JSP with scripts written in any of the .NET languages, normally Visual Basic (VB), C# (Microsoft's alternative to Java) or F# (a new functional programming language). Higher-level frameworks are available. Development is expected to be performed in a proprietary develpment evenironment such as Visual Studio.
I suspect .NET is less complex than Java EE but offers correspondingly fewer benefits. Like Java EE, it is most commonly used in large organisations. Using either Java EE or .NET seems to require developers to study a bookshelf full of manuals and guides.
Ruby on Rails was developed to make Web development easier and faster. It addressed the low-level of PHP and the complexity of Java EE and ASP.NET. It aims to be "a framework that makes it easier to develop, deploy and maintain Web applications" (Thomas and Hansson, 2007).
Ruby on Rails (or Rails for short) is based on two main principles:
Systems developed with Rails are required to observe a model-view-controller (MVC) architecture. This architecture clearly separates underlying data (model), user interface (view) and application logic (controller).
Rails has developed an enthusiastic and supportive user community and is becoming increasingly used for small to large applications, including (initially) twitter.
Its design takes advantage of the excellent object-oriented programming language Ruby.
Arguably, Rails takes principle 2 too far. This imposes restrictions on developers, which make it difficult to develop applications whose structure differs from the one expected by the Rails developers.
Django is a successor to Rails, which uses the excellent programming language Python instead of Ruby, and which attempts to offer developers greater flexibility than Rails does.
Django is used by many major newspaper publishers and was recently adopted for use in the Google App Engine, a service that provides both free and paid Web hosting to users.
In 7401ICT, we shall use Django exclusively as the platform for developing Web services.
Like Rails, Django is also based on the model-view-controller pattern, but is more flexible (and uses different terminology). It uses "model" to refer to the underlying data, "templates" to refer to the user interface, and "views" to refer to the application logic. Perhaps we should say it uses the MVT pattern.
A Django "project" is a directory, e.g., guestbook,
that contains:
settings.py - general configuration file
urls.py - a mapping from URLs to views (Python functions) that
process themtemplates - a set of HTML templates
Every "application" directory, e.g., messages, contains:
From the "project" directory, you can give the command
$ python ./manage.py runserver [hostname:port] &
and then you can give URLs of the form:
http://hostname:port/messages/
to run the guestbook application using the development server.
(The default port is 8000, but - in our context - each student will need to
use a different port.) We'll describe later how to use the production
server.
The real advantage of Django over PHP is that it improves development, deployment and maintenance by making implementations shorter, better structured, and more flexible. Below, I give examples to support this claim. Many of these examples are specific cases of the DRY principle.
Django uses an object-relational mapping (ORM).
There is no need to create SQL tables, corresponding to the Django models, as this is done automatically.
There is no need to define SQL insertion and selection queries, as simple Python function calls are translated into SQL queries.
There is no need to sanitise user input to avoid SQL-injection attacks, as this is done automatically in translating Python function calls to SQL queries.
In the simplest case, we can construct a form directly from a model, and construct code to process form data directly from the form. So there is no need to repeat the list of fields in the model, in the SQL "create table" statement, in the form definition, in the SQL "select" statement, and in the script that processes the results of the SQL "select" statement.
It is also possible to automatically validate user input depending on the field type.
It's possible to automate the display of a list of objects, with links to details of each object. By adding a single attribute, the list will be displayed one page of results at a time.
By editing the configuration file, it's automatically possible for registed users to have superuser access to all application tables. It's further possible for such superusers to assign authority to perform selected operations to selected users.
Django provides support to greatly simplify the management of user registration, logins and permissions.
User input is automatically sanitised before being used in database queries or in HTML documents, to prevent cross-site scripting attacks and SQL injection attacks.
There are any many other time-saving conveniences in Django...
Django is not as widely used as PHP, Java EE, .NET or Rails. However, its use is growing rapidly, it is based on the same principles as these other frameworks, and a sound knowledge of Django makes the transition to any other modern framework relatively easy.