mvalente.eu/mv.*Heavy Mental Stuff

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Thin Server Architecture
Date: Sun, 16 Mar 08 23:30:21 GMT

As usual on the Internet, the same idea seems to bubble up
from several people all at once.

For the last 2 years or so, I’ve been thinking a lot about
web app architecture and app server frameworks. My thinking
has led to an approach that I’ve been trying to explain in
some previous posts.

But now Peter Svensson has given it a name:

Thin Server Architecture
http://unclescript.blogspot.com/2008/02/thin-server-architecture.html

Sounds like a nice buzzword and a nice TLA: TSA.

Peter has gotten together a group of people (me included) and
kickstarted a working group, already working on putting up some
clustered information in a place of its own:

Thin Server Architecture Working Group
http://www.thinserverarchitecture.com/

Meanwhile, while researching on the subject, I got a couple of
aditional links that are important for our thinking and for my
continuing posts about a future web framework:

How Nexaweb Works
http://www.nexaweb.com/home/us/index.html@cid=2295.html

Developing Rich Internet Applications with Declarative Markup
http://rockstarapps.com/wordpress/wp-content/uploads/2007/08/java-basedria.pdf

Slides 6-9 are especially interesting. In slide 9 you can easily see
what TSA is all about: its represented by the 4th stack, the one where
the app container has only UI stuff (view+logic) and server side you
only have biz logic and data.

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: A Future Web Development Framework – Template View
Date: Wed, 13 Feb 07 18:43:21 GMT

I should start this post (hopefully the last detail before
going into a synthesis post) by remembering some words from
my initial post in this series:

“Meanwhile, lots of stuff has come up: Django, Pylons,
TurboGears, Ruby on Rails, you name it… I dont want to
go over extensively why I dont like any of them, but let
me just state this: if the “framework” allows for the
mixing of layout/design/HTML with code/programming, for
me is enough of a disqualifier.”

The problem, as I stated before, is that current frameworks
stuff a lot of power into the templating languague. And that
is not only unnecessary but its a bad, bad thing.

If you dont understand why 1) embedding Python/PHP/Ruby/etc
in HTML is bad or 2) outputting HTML from Python/PHP/Ruby/etc
is worse, I’m not even going to try and explain. Not only
because that means you are clueless but also because someone
else has done it better that me:

http://www.stringtemplate.org/about.html
http://www.cs.usfca.edu/~parrt/papers/mvc.templates.pdf

Now the problem is that although I agree with this paper and
the rules set forth in it, I disagree with the implementation.
Allowing for “$var” statements in the templates is still, in
my view, some form of entanglement and breakup of the MVC pattern.

What would be the best way to do it? Well look no further than
some already existent template engines:

PyMeld
http://entrian.com/PyMeld/

HTMLTemplate
http://freespace.virgin.net/hamish.sanderson/htmltemplate.html

Meld3
http://www.plope.com/software/meld3

They allow for the insertion of values (data binding) through
the use of HTML element attributes. Both HTMLTemplate and Meld3
create new attributes (”node” and “meld:id” respectively) but
PyMeld does it better: it simply uses the “id” attribute to do
value replacement. This is, in my opinion, the best way to do
a templating system. And there’s no need to make it “powerful”
through the introduction of loop control and other stuff like
Genshi or Zope Page Templates.

Does this mean that I have found my perfect templating engine?
No. You see, these are serverside templating engines and the
template/view should be implemented at the client level, if you
want to adhere to the MVC pattern.

What I would like is something like these templating engines:

Ajax Pages
http://ajax-pages.sourceforge.net/

Javascript Templates
http://www.fishwasher.com/jst/

Trimpath Junction Javascript Templates
http://code.google.com/p/trimpath/

Unfortunately all of these engines make the usual mistake of
introducing new syntax to HTML (the “$var” problem or variants
thereof instead of relying on the existing HTML markup, namely
the “id” element markup.

To sum it up, what I would rely like is a Javascript implementation
of Meld.

Aditional templating functionalities that might not be provided
through the use of the “id” attribute (although I think that it
would be all that is needed) could be added through the use of
aditional semantic potential to existing HTML markup with Javascript.
See for example Inc, for JS code include (although I would prefer
for this to be done through the implementation of the “src=” attribute
for DIVs.

http://johannburkard.de/blog/programming/javascript/inc-a-super-tiny-client-side-include-javascript-jquery-plugin.html

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: RESTful DB and App Servers
Date: Sun, 09 Dec 07 19:40:21 GMT

Like I said before, most of the week’s linkage
has had to do with CouchDB or, generally speaking,
to do with JSON-formatted REST-accessed database and
application servers.

http://mvalente.eu/cat.cgi?JavaScript%20in%20All%20of%20the%20Tiers
http://del.icio.us/mvalente

CouchDB is sure fun to use, but its Erlang dependency just
pisses me off.

http://couchdb.org/
http://code.google.com/p/couchdb/
http://jan.prima.de/~jan/plok/archives/108-Programming-CouchDB-with-Javascript.html

The way that I’ve been thinking about a new web development
framework indeed involves a REST interface and JSON or XML
formatted data. But I just think that a) you might as well
standardise the programming languague used at all tiers (and
that means Javascript) and b) you should have REST/JSON as
interface not only to the DB tier but also to the logic tier.

Sure enough there are some alternatives if you want to go
that way.

Basura DB (Python based JSON/REST DB implementation)
http://www.intertwingly.net/code/basura/
http://intertwingly.net/blog/2007/09/24/Tests-Id-Like-CouchDB-to-Pass

Persevere (Java based)
http://code.google.com/p/persevere-framework/
http://www.ibm.com/developerworks/web/library/wa-aj-objmap/

Robaccia (Python based web framework includes a RESTful JSON server)
http://bitworking.org/news/restful_json_server

JSONStore
http://taoetc.org/97

DBSlayer (HTTP/JSON interface for MySQL databases)
http://code.nytimes.com/projects/dbslayer

But aditionally to databases, I’m thinking more in line with
the proposed RADAR architecture of having the REST interface
also available for the app server. There arent as many options
here…

The RADAR Architecture
http://pragdave.pragprog.com/pragdave/2007/03/the_radar_archi.html

Naked Objects
http://www.nakedobjects.org/tutorial/index.shtml

NetKernel
http://www.theserverside.com/news/thread.tss?thread_id=47695
http://www.1060research.com/netkernel/

And sure enough all of the above is typically Java-based (yuck!),
with some other languages thrown in (Python, Erlang), but none
goes all the way and standardises on Javascript as the implementation
language and the glue language. Apparently I am not the only one
considering this idea…

Alt Framework
http://marcello.cellosoft.com/projects/alt/

Simile’s Exhibit Framework
http://simile.mit.edu/exhibit/

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: MVC, MVP, PAC and Others
Date: Thu, 22 Nov 07 23:42:21 GMT

I’m trying to wrap my mind around Model-View-Presenter,
Presentation-Abstraction-Control and other UI architectural
patterns, so I might as well warn you that my brain might
explode…

http://ctrl-shift-b.blogspot.com/2007/08/interactive-application-architecture.html
http://www.martinfowler.com/eaaDev/uiArchs.html

Definitely stuff to think about regarding a future web
app framework.

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: SOFEA and the Future Web Development Framework
Date: Wed, 21 Nov 07 01:38:21 GMT

“SOFEA is an architectural style which in its definition
criticizes (and manages to make a very good definition of)
the current web frameworks, and (as they say) “none of them
satisfies”.

http://unclescript.blogspot.com/2007/11/coming-frameworks-sofea-jda-squirrel.html

This is what I have been talking about in my posts about a
future web framework. The guys that are proposing SOFEA make
a very good technical argument in their paper and presentation.

http://wisdomofganesh.blogspot.com/2007/10/life-above-service-tier.html

If you pay particular attention to page 26 in the paper (about
how the Service Oriented Front-End Architecture model should work)
you will see that it is precisely what I’ve been talking about.

David Coffey also seems to have come to the same conclusions. And,
like me, that it all converges into Javascript. He’s building Triad,
a Javascript MVC framework.

http://blog.davidadamcoffey.com/4/triad-a-javascript-mvc-framework

Compare his diagram:

http://blog.davidadamcoffey.com/wp-content/uploads/2006/11/mvc2.png

With my previous post:

http://mvalente.eu/cat.cgi?A%20Future%20Web%20Development%20Framework%20-%20Architecture

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: A Future Web Development Framework – UI Controller
Date: Sat, 03 Nov 07 21:03:21 GMT

There is even less to say about the controller part of
the framework. We dont even have to deal with religious
wars regarding what language to use. Your option is Javascript
and nothing else. Anyone thinking “what about Java” or “what
about Flash” (and current iterarions, Air, JavaFX, etc) will
have to explain what is the justification to have a VM/interpreter
available (the browser) and to start yet another VM/interpreter
(JRE or Flash plugin).

Nonetheless there are a couple of points that should be made.

A future web framework would have to guarantee that the controller
part, in Javascript, doesnt allow for business-logic embedded in
the controller code and doesnt allow for obtrusive Javascript (in
particular inline code)

The right way to do it would be to make declarative the use of
the UI behaviour the same way that, using CSS, we did away with
inline styles.

An obvious candidate would be, of course, Behaviour

http://bennolan.com/behaviour/

But there are other options. One is event:Selectors

http://alternateidea.com/event-selectors/

(although its based on Prototype.js and much prefer JQuery.js)

Another would be XForms and any Javascript implementation of
it (ie. not having to depend on the browser implementing the
standard)

http://ajaxforms.sourceforge.net/
http://www.formfaces.com/

Here though we start getting into a lot more than the UI
behaviour, namely we start getting into template/view stuff
and I want to deal with this in a separate post.

Its not always easy to define where the UI/controller part
is different from the UI/view/template part. I have a button
that takes 2 fields and computes some function. Should that
be done in the controller or in the template? I have a button
on a menu that takes me to another application screen. Where
should that be done, in the controller or in the template?

(just for the record, the first one is template level since
it has to do with data processing, data binding and business
logic, and the second one is controller level since it has to
do with application flow)

But it is difficult to define and decide and it is one of the
major problems with current web frameworks since it leads to
spaghetti code mixing controller stuff and model stuff. A new
web framework should be able to stop or minimize this problem.

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: A Future Web Development Framework – Application Logic
Date: Sun, 07 Oct 07 21:21:21 GMT

There really isnt much to say about the middle component for
a 3-tier architecture (the business logic). You can of course
get into religious wars about which language it should use. This
is the component that usually generates all the talk precisely
because of the language issues. When discussing Django, Rails,
.NET, etc, people arent really arguing about what the application
(logic) server should do or what its architecture should be: they’re
just arguing on whether Python is better than Ruby is better than
Rails is better than PHP…

Me, although I currently favour Python, I am more of a syncretic
integrator: whatever solves the problem faster without spending
too much of my brain processing time (this immediately eliminates
Java.. LOL :-). Thats why I like some of the more recent options
available like Virtuoso or ActiveGrid that allow you to program
the business logic in a variety of languages (either choose one or
use all at the same time).

http://www.activegrid.com/
http://www.openlinksw.com/virtuoso/

Virtuoso, for example, by including the VM runtime for several
languages, is able to run code in Java, .NET, Python, Perl or PHP.

http://www.openlinksw.com/virtuoso/architect/vtechnical.htm

(nice graph there, maps some of my ideas nicely)

Nonetheless there are two points worth exploring about the logic
layer, whatever the language of your choice: how the logic layer
interfaces to the storage layer and how the logic layer interfaces
to the presentation layer.

The first problem I have already referred to in my previous post
about storage. Even though the usual interface is the SQL language
(due to the fact that 99% of the times the storage layer is implemented
using a relational database), nowadays most programming is being done
using an object-oriented language. Hence the object-relational impedance
problem and the proliferation of object-relational mappers (ORM), which
have become especially popular due to its use in Ruby-on-Rails ( more
precisely ActiveRecord).

As I stated in my previous post I dont think that this is the way of
the future. The indiscriminate use of the relational/SQL database is
way past due in an object/document/unstructured/hierarchical oriented
world and object-oriented databases should become the norm. On the other
hand I do have to admit that having my data closed down in some format
which is proprietary really bothers me a lot. Since I wrote my post about
storage, CouchDB has entered the radar of many people and it seems to
contain the basis for what is needed: a OO database that stores its
objects in a simple text format (JSON) with a REST interface.

http://couchdb.org/

The second problem (logic-to-presentation interface) actually mimics
the logic-to-data interface and it also has something to do with a later
issue (templating or view/controller). As in the logic-to-data interface
problem, where the logic level (ex: Perl or Python) currently embeds a
lot of data stuff (ie. SQL), the logic-to-presentation interface also
suffers from a lot of confusion, especially when you have templating systems
that allow you to mix code and presentation. PHP is a particularly bad
example of this problem.

This could be solved by explicitly denying the logic level to output
any kind of presentation code. Presentation level stuff (ie. templates
or view/controller) would be in charge of doing any presentation stuff,
getting all the data/logic from the applicatin server (through REST calls,
with data in JSON, XML, SOAP, whatever), with the application server being
explicitly prohibited from outputing any kind of HTML markup code (it just
wouldnt be taken into account or rendered).

What comes to mind as an example is an application server, using whatever
language, that would behave exclusively like TurboGears/CherryPy with the
jsonify decorator: you could call any URL in AJAX form and you would receive
results in JSON (or any other format), no HTML allowed.

http://docs.turbogears.org/1.0/JsonifyDecorator

Although I see no problem with the same server delivering both page
templates (ie. view/controllers) and logic (they are both HTTP after all),
that implies some discipline. An actual separation of concerns would
imply separate servers: one would only serve static pages (ie. templates)
and could be a standard HTTP server (Apache, etc) and the other would
only serve the processing results for received requests.

Event though I’m a bit agnostic regarding the choice of language for
your application server, I do have to state my opinion regarding current
trends and what I see as being the future endgame and the best choice.

Everyhing points towards a Javascript future.

http://steve-yegge.blogspot.com/2007/02/next-big-language.html

This is probably unpopular to say and there will be several voices
talking about Javascript’s speed. But the fact is that it doesnt make
much sense to use a language at logic/application level and then to
use another at presentation/template/view/controller level. Especially
when you take into account the current AJAX and JSON trends. You just
create aditional impedance when your doing stuff in Javascript at
presentation level (lets say field validation for example) and then
you’re encoding form fields into XML or POST fields, requesting some
processing from the application server which has to unmarshall the data
into whatever its format is, process it and remarshall the data back
into XML or JSON for delivery to the presentation layer.

In the last couple of years Javascript, which was available server
side back in the Netscape days (it was called LiveWire), is making
its comeback in the application server layer with a bunch of new
developments like OpenMocha, Phobos or Helma.

http://helma.org/
http://openmocha.org/
https://phobos.dev.java.net/

So, to sum it up: a future web development framework would include
an application server that exclusively served webservice requests
preferably in a REST way and in JSON format, explicitly barring any
output of presentation/HTML code. Although it could work in any
language, Javascript would probably be the best option given current
trends.

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: A Future Web Development Framework – Storage
Date: Sun, 26 Aug 07 15:55:21 GMT

The way I see it, there are two problems regarding
storage in web frameworks that are orthogonal to
each other:

– one problem is the fact that we need storage at
each level of the 3-tier architecture and/or at
each component of the MVC pattern. We need to store
the presentation templates/views, to store the
logic/model and to store the data itself. Right now,
and this is one of the problems of current frameworks,
different types of storage are used at different levels.
Usually templates and code are stored using files at
filesystem level and data is stored using a relational
database.
This generates two subproblems: one is that you have
a disconnect between types of storage used (and dont
tell me that you could use the relational database to
store scripts and templates; you just look silly) which
gets even more ugly when the data you want to store is
not structured/relational (files? images? documents?).
If you suggest using blobs in the relational database
you wont just look silly but also stupid.
The second subproblem is the disconnect you get from
using a relational database to store the object oriented
models from the MVC pattern. The infamous object relational
impedance, usually solved by the use of OR-mappers (that
only go so far because they store objects properties but
not objects methods.

– the second major problem is the type of storage to
use, preferably at all levels of the 3-tier architecture.
As I said before, usually filesystem files are used for
templates and code and relational databases for data.
But this is just “legacy” thinking and it generates
impedance problems, not only between MVC/OO thinking
and relational storage but also between managing files
and managing tables.

What should or could be the way to tackle these two
problems? Well, I think that using a single uniform
storage would be the way to do it. You either decide
that you’re going to use files all the way, relational
database all the way or whatever other option all the
way.

I think that relational is out of the question. Not
only is it stupid to store templates and/or code in
this way, but you also get the OR impedance problem
and the “binary files in blobs” problem.

So its either filestorage for all or a second option.

Filestorage is quite atractive. Templates are already
there, code is there, easy to store documents/blobs…
The problem is structured data, usually stored in
relational databases.
One option to solve this would be to use a file level
abstraction of relational storage. Plan 9 stuff comes
to mind; an interesting option would be the use of
libferris, since that would also bring aditional file
abstraction views of several resources.

http://witme.sourceforge.net/libferris.web/

Doing “ferrisls -0 pg://localhost/xmldotcom2007/customers”
to get a listing of the Customers table in PostgreSQL is
quite cool…

Another option would be to use plain old files to
store data. This data could be stored formatted in XML
(as is now in fashion), although with current interest
in REST and JSON, it comes to mind that plain old RFC822
formatted files would fall in quite nicely (even the
separator ‘:’ is the same…).

For this option I can think of at least two tools
that might come in handy. APE and plain old IMAP/NNTP
storage…

http://hathawaymix.org/Software/Ape
http://deflexion.com/messaging/
http://pessoal.org/papercut/

Filestorage would have aditional benefits of not being
opaque, easily accessed by other tools. There would also
be the benefit of being able to use “normal” tools (like
vi) or more “advanced” ones (like versioning and backups)

The second option for storage: object oriented databases.

Their time is now more that due. Early ones had huge
performance problems and didnt have the SQL version du
jour. But there are now some OO databases that have quite
good performance, possibility of XML and document storage
and even SQL queries.

Some standard relational databases, like PostgreSQL,
already have the OO usage option.
Another possibility is the use of native XML databases
like eXist.
Finally, the way that my longtime favourite Zope has
chosen, the use of a native OO database (ZODB in Zope’s
case)

I can tell you that I’ve been a bigger fan of Zope/ZODB
in the past. At the end, the fact that ZODB is opaque (ie.
you can only get at your data using Python as tool)
actually becomes a problem. You get nice “things” like
acquisition, persistence, versioning, etc but you miss
out on accessibility (using vi, doing a grep on all your
code, etc). Unfortunately the development of APE and
related tools seems to have stopped, since that would seem
to be the perfect solution.

http://www.zope.org/Members/jdoyon/svnapezope

In conclusion: a future web framework would have to
have a single storage system for all its components. This
could either be a filesystem based system providing an
interface acceptable to the code level (SQL? XML?)
or an OO database with storage viewable as filesystem
objects. An important thing: the interface provided to
the code level could have SQL, SPARQL, XPath, whatever,
as options; but as time goes by its becoming clearer that
a REST interface is obligatory.

http://www.25hoursaday.com/weblog/2007/07/13/GoogleBaseDataAPIVsAstoriaTwoApproachesToSQLlikeQueriesInARESTfulProtocol.aspx
http://exist.sourceforge.net/devguide.html#N1021D
http://phprestsql.sourceforge.net/

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Zope Replacement
Date: Tue, 14 Aug 07 00:28:21 GMT

Nearly 8 years later, even though I’m dissatisfied with the path
Zope took (in particular version 3), I’m still searching for something
that is better, since all the other web frameworks suck.

http://groups.google.com/group/pt.comp.programacao/msg/01af20cd35371370

http://zopereplacement.wikidot.com/

Turbo Gears comes close to what I would like to use; but its
still missing so many important things that I prefer to keep
looking and to brainstorm on how to do right a future web
development framework.

http://www.turbogears.org/

http://www.reahl.org/wfmwiki

— MV