For the last 2 years or so I have been harping about a Future Web Development Framework and about how Javascript Will Save Us All. And even 2 years before that (4 years ago) I had actually come to the conclusion that Javascript would be the next big language and the one that would invest my time in for the next 10 years (2005-2015).
Aditionally, 2 years ago I hinted at the need and possibility that Javascript should be used at in all of the tiers of aplication development. And 1 year ago, at Codebits, I restated the need for a totally Javascript-based, REST-accessible, JSON-formatted web development framework. I tried to challenge people at Codebits to work in it, but no such luck. I also tried to start developing it but it was a no go: I dont have as much patience for coding as I used to (age is a bitch) and the context for serverside Javascript development just wasnt right (lack of standards, lack of OS API access, etc). And even having some Javascript interpreters (like jslibs which is what I was using at the time) with access to stuff like files and sockets wasnt much of a help: its not very much fun to try and reinvent the wheel writing a webserver from the (socket) ground up.
Well, things have changed a lot in this past year. The ServerJS/CommonJS group has been doing a lot of work on standardising a unified JS API. And there’s been a lot of work done on top of several Javascript interpreters/compilers/VMs (like Spidermonkey, Rhino, V8, etc) to provide for basic system level functionality. Plus a lot of Javascript-based basic blocks have been developed (like Narwhal, Nitro, Node, Awesome, Lawnchair, DjangoDE, PURE, etc) that allows us to get more concrete about what is needed and what can be done.
So its time to once again tell the Lazyweb what I think is needed or (better yet) what I want in a new web development framework that lasts for 10 years. Here goes:
- I want it to be Javascript based. All of it. No Erlang, Python, Ruby, etc in sight. I want the frontend/View/presentation in Javascript. I want the appserver/Controller/logic in Javascript. I want to program the appserver in Javascript. I want the DBserver/Model/data in Javascript.
- I want all the Javascript code to be compliant with the CommonJS standards. The code should be portable to any of the existing engines (Spidermonkey, Rhino, V8, flusspferd, Jaxer, Helma, etc) although I would touch anything Java-based with a long pole, so please make it work on Spidermonkey or V8.
- I want all the data to be JSON formatted and REST-accessible. The data on the database should be stored as JSON data and accessible via REST. The appserver API should be REST-based, accept JSON-formatted parameters and return JSON-formatted results. The templating system should accept JSON-formatted data for rendering the view and make REST JSON-formatted calls to the appserver.
If this isnt clear enough, let me refer some analog components that are written in other languages and what is already available in JS:
- for the database I want CouchDB but written in Javascript. Awesome or Lawnchair might be starting points. The endpoint would be something like Persevere but without the Java/Rhino dependency. Maybe Pintura is *it*…
- for the appserver I dont have the right analog. Of course that it must support JSGI. But *the* most important thing: it can not and must not allow the generation of HTML and the return of HTML to the browser/client! All methods/functions should only accept parameters and return results in JSONified format. Maybe NitroJS could be *it*. If it threw away the sucky template format. Completely. No templates on the serverside. No serverside HTML generation. Pure HTML is the new template standard (see below).
- URL routing should use the object hierarchy on the appserver as default. Explicit URL routing (static or using regexp) would have to be done explicitly.
- for the presentation layer I want pure HTML+JS. The browser should be the only component responsible for rendering templates. The clientside part of the framework should make all calls to the server using REST/AJAX and providing JSON-formatted parameters. No GET or POST parameters. It should only accept JSON-formatted results. No HTML. Standard HTML is the only templating system needed. Page elements (DIV, SPAN, etc) should have an “id” as an identifier for serverside databinding. If a page has a tag/field with “id=xpto” then that tag/field should be filled with the contents of the “id” key returned from the server in a JSON string. A good starting point would be PURE, but without using “class” as the databinding identifier (“id” should be the standard).
- I want the appserver to have features that Zope has had for years: I want acquisition (aka prototype-based programming, aka JS object-orientedness). I want a through-the-web management interface. I want ZClasses. I want automatic form generation for classes/objects and an automatic management (CRUD) interface for objects.
So, how’s that? It’s Xmas anyway, so I can ask for whatever I want. Gimme, gimme… Or I can start working on it… Who’s with me? I can be the Main Bitcher, Overall Architect and Project Manager.
may i add extJs to the soup? for the client side, of course. then it would communicate with the server using JSON and REST.
join that with html 5 local storage (heard about it, still havent read about it) and the chrome extension technology that allows you to “access” the pc with js and we could reach the sweet spot between web and local applications. still, that chrome thing reminds me a lot of IE browser helper objects aka disaster waiting to happen. we’ll see.
i need all that too. now. im lazy and that would take a LOT of redundant work out of me. oh well…
>may i add extJs to the soup?
>
Sure. But you can make REST/JSON with other clientside frameworks. Plus extJS is a bit heavy on UI stuff.
>join that with html 5 local storage
>
Yes, that would come in handy for clientside persistence. But you can already do it without HTML (ie. Google Gears)
— MV
Take a look at my stab at a HTML+JS templating engine. It’s at http://github.com/chbm/deltatemp .
It takes a JSON object and applies it to an html template where the variable names are enconded in classes. It can do straight substitutions and more complex stuff like tables using arrays and dictionaries.
I honesty can’t remember why it uses class to map variables, probably run into some problems with generated elements, but I’m not very keen on that and might just rewrite it :)
>It takes a JSON object and applies it to an html template where >the variable names are enconded in classes.
>
Classes are not the right way to do it.
And I could scream at those “$” inside class names….
“id” is what should be used. Its what its for.
— MV
Did you see this?
http://nodejs.org/
Is this the Node you talk about?
>Is this the Node you talk about?
>
Yes it is.
— MV
I started working on a small client-side (library indepent) event-based JavaScript micro-framework a couple of weeks ago.
The base is the Observer Pattern and the goal is to have loosely coupled client-side code. It’s still very raw and incomplete (http://github.com/mlouro/olivejs). I’m still figuring out what the best way to handle urls and templates will be.
Will you be attending codebits this year? I’d be interested in listenin and sharing a few thoughts on the topics you mention.
> Will you be attending codebits this year?
>
Yes, I will visit a couple of times though I dont intend to spend a lot of time there, to do presentations, projects or competitions.
We can of course meetup and talk a bit
— MV
Have you looked at jstal (http://code.google.com/p/jstal/wiki/usage) or jtemplates (http://jtemplates.tpython.com/) ?
I can’t see how only id is enough for a template system. Perhaps for a placeholder template is enough, but how would you to iterate through a list, or adapt to missing data?
>Have you looked at jstal (http://code.google.com/p/jstal >/wiki/usage) or jtemplates (http://jtemplates.tpython.com/) ?
>
Believe me, I have a PhD in template systems :-)
Me and TAL go back a *long* way ( http://copilotco.com/mail-archives/zope.2001/msg15132.html ), TAL originally comes from Zope.
TAL and Jtemplates have *lots* of problems. 1st they are generated serverside. Which is wrong. 2nd they mix logic and presentation. Which is not only wrong ( http://www.cs.usfca.edu/~parrt/papers/mvc.templates.pdf ) but also a kitten dies everytime a template system is done like that. 3rd they make you learn a new pseudo-language. I could go on…
Everytime I see a $ in a template system, it gets discarded pretty quickly.
>I can’t see how only id is enough for a template system. >Perhaps for a placeholder template is enough, but how >would you to iterate through a list, or adapt to missing data?
>
No “template system” hidding in “id”, please! No!!! You’re doing it wrong!
An HTML has an “id”. That “id” is for presenting a value returned from the server, JSON-formatted, where an key=”id” is present. Pretty simple. Its what ID is for ( http://www.w3.org/TR/html401/struct/global.html#h-7.5.2 )
If returned data from the server is missing an element named “id” then the HTML is not to be filled.
If the “id” element returned from the server as is itself a JSON-string with multiple keys, then the client side template system iterates through the multiple keys and reproduces the HTML element N times. Check out PyMeld ( http://entrian.com/PyMeld/ ), they’re doing it right. Except its serverside and in Python. That needs to be done right: in Javascript and clientside :-).
— MV
I figure you would have know TAL, having worked on Zope.
I don’t want to seem ignorant, but there are some stuff I can’t grok yet.
JSTAL and jtemplates work client-side: at least I’m using jtemplates client side.
How would you deal with non-existent data? Sometimes you can’t just leave the tag empty or it may mess up with the browser. There is a need for presentation logic.
PyMeld seems nice from a quick glance, but I spot a potencial problem: duplicate ID on the output, which is not right based on W3 specs. This is the reason you see classes being used so often.
I may be wrong, but please share some insights with the rest of us :)
I liked your starbuck presentation last year, but I settled on some silly thing for my project (in JS, though).
There seems to be some interest in the subject. How about setting up a mailing list or google group for further discussion?
>I don’t want to seem ignorant, but there are some stuff I >can’t grok yet.
>
:-) I still cant understand about 1/3 of it :-)
>JSTAL and jtemplates work client-side: at least I’m using >jtemplates client side.
>
Yes, you’re right. Its just that the tests I made with JSTAL were serverside. JSTAL does seem to work clientside.
JTemplates are in fact clientside, my mistake.
The problems I have with both is, once again, mixing logic/code in presentation/layout. And having to learn a new “language”/”syntax”.
JSTAL would be OK. Just make it work with “id” instead of all that “tal:zbr”,”tal:xpto” nonsense :-)
>How would you deal with non-existent data? Sometimes you >can’t just leave the tag empty or it may mess up with the >browser. There is a need for presentation logic.
That presentation logic should be automated as much as possible. Namely it should be declarative and not imperative ( see Behavior.js http://juixe.com/techknow/index.php/2007/05/02/unobtrusive-javascript-with-prototype-and-behavior/ and BehaviorSheets http://code.google.com/p/reglib/ )
I dont see what the problem is with non-existent data. If an empty tag creates problems with the browser, you’re doing it wrong. And in that case you could deal with that by making the clientside template engine make that tag/element go “display: none;”
>PyMeld seems nice from a quick glance, but I spot a >potencial problem: duplicate ID on the output, which is not >right based on W3 specs. This is the reason you see classes >being used so often.
Yes PyMeld is doin it wrong. But using classes is even worse. I’d rather have duplicate IDs and overlook the W3 specs. Which would probably make sense, since if several elements have the same ID they are supposed to be the same, have the same layout and the same values.
Another way of dealing with that would be by adding a numbering scheme to the ID (ID1, ID2, etc) that must be repeated.
Or one could just go HTML5 custom-data-atributes on the mofo… :-)
>There seems to be some interest in the subject. How about >setting up a mailing list or google group for further >discussion?
>
That would be http://groups.google.com/group/sofea except for the fact that it has had no msg flow in the last few months (except for the inevitable spam)
— MV
I had to look three times before I really believed it. “2009/11/26”. Yes: November 26th, 2009.
MV: “I dont see what the problem is with non-existent data. If an empty tag creates problems with the browser, you’re doing it wrong. And in that case you could deal with that by making the clientside template engine make that tag/element go “display: none;””
So I should just leave in my html and hide it with JavaScript? Are you kidding? Never heard of semantics?
MV: “I’d rather have duplicate IDs and overlook the W3 specs.”
Aargh. How can anyone take you seriously?
If you only knew how many kittens you have killed by the way you program. The worst thing is you’re not aware of it.
And about declarative logic: it seems jQuery is the hot thing now.
And your comment system rips out the html instead of encoding it.
I wrote:
“So I should just leave (h1)(/h1) in my html and hide it with JavaScript? Are you kidding? Never heard of semantics?”
Replace “(” and “)” by you-know-what.
>I had to look three times before I really believed it. >“2009/11/26″. Yes: November 26th, 2009.
>
I dont get it. What is the issue? Care to explain?
>And your comment system rips out the html instead of
>encoding it.
>
Its standard WordPress go and complain there.
>“So I should just leave (h1)(/h1) in my html and hide it
>with JavaScript? Are you kidding? Never heard
>of semantics?”
>
If it has no content is it displayed? No. But it is still on the page content. If it has content then the templating system wouldnd hide it and present it.
>Aargh. How can anyone take you seriously?
>
Whoever wants to do so. I’m not looking for aproval.
>If you only knew how many kittens you have killed by the >way you program. The worst thing is you’re not aware of it.
>
Actually, I am :-)
And even though I do, that is not the issue at hand, is it?
— MV
I could try to work it out in Haskell, mixing HappStack (a low-level *FAST* application server) with HJS (JavaScript parser and interpreter).
Zope is nice, the component architecture is definetly a MUST http://www.muthukadan.net/docs/zca.html .
Also BFG should not be disregarded as it entails Zope’s 10 years of industry and programming experience.
>I could try to work it out in Haskell, mixing HappStack (a >low-level *FAST* application server) with HJS (JavaScript >parser and interpreter).
>
Holy mother of God! An unkown huge amount of kittens have just died…
I wouldn touch Haskell with a pole. A long one. JS has all the functional goodies that are needed. Now if you rewrote HappStack in JS and HJS in JS… ;-) You might have to rewrite Haskell in JS, but that would be a minor detail :-)
>Zope is nice, the component architecture is definetly a
>MUST http://www.muthukadan.net/docs/zca.html .
>
Zope *was* nice until Zope 2.X and until Jim Fulton decided to explode everybody’s brain with the move to Zope 3.X and ZCA and all the other academic complex bits. If I wanted to complicate, Java would be a better option.
>Also BFG should not be disregarded as it entails Zope’s 10
>years of industry and programming experience.
>
Yeah. BFG is basically Zope. With all that entails. Remember, I have used Zope for the last 10 years. Nothing comes close. But it just doesnt cut it anymore.
BFG would be nice. If it wasnt serverside. And written in Python. And recquiring learning a new “language”. And being inconsistent with MVC.
Using BFG? Only the one in Quake…
— MV
> Zope *was* nice until Zope 2.X and until Jim Fulton decided to explode everybody’s brain with the move to Zope 3.X and ZCA and all the other academic complex bits. If I wanted to complicate, Java would be a better option.
MVC has reached its limits, it is just a pain to keep repeating huge amounts of code to create a workable framework (yeah, “DRY, just use our package that repeats everything for you”).
Adapter+interface is a much better approach to web frameworks that eventually is going to win. Not wanting to learn it doesn’t make it bad, it just makes you a lazy-ass person :)
> BFG would be nice. If it wasnt serverside.
I thought we were talking about serverside here…
Btw, im not sure what is the current state with JS libs, but definitely they are not as mature as Java, Python or even Haskell libs.
This is the cornerstone with going serverside with JS -> reinventing the wheel.
>MVC has reached its limits,
>
MVP http://en.wikipedia.org/wiki/Model-view-presenter
>I thought we were talking about serverside here…
>
We are. Except for the templating system.
>Btw, im not sure what is the current state with JS libs,
>but definitely they are not as mature as Java, Python
>or even Haskell libs.
>
Sure they’re not. Feel free to keep using the mature solutions that mix languages, code, presentation, etc.
— MV
>
Give me coherent DSL’s built on a sane stack – I don’t give a crap what language it’s built in.
JavaScript is a cool language, but persuing it as a solution for all levels of the stack is essentially just pointless mind-wankery. We have technology on the server side that works good enough already.
Besides; learning multiple languages is good for your brain. :D
>Give me coherent DSL’s built on a sane stack – I don’t give a >crap what language it’s built in.
>
I do. I would hate to use a coherent DSL written in COBOL.
>JavaScript is a cool language, but persuing it as a solution for >all levels of the stack is essentially just pointless
>mind-wankery. We have technology on the server side that >works good enough already.
>
You just have to use it then. Its a free world after all.
>Besides; learning multiple languages is good for your brain.
>
One could say that learning multiple languages just for the sake of your brain is pointless mind-wankery…
— MV
>Give me coherent DSL’s built on a sane stack – I don’t give a >crap what language it’s built in.
>
why aren’t we all coding in BrainFuck then ? :)
JavaScript is going to experience a much broader usage in the future, predicting that was kind of obvious (specially since gmail came to the show).
This page might also be interesting http://en.wikipedia.org/wiki/Server-side_JavaScript
Also, i’ve been told that years ago Broadvision had a system using Netscape’s server side Javascript, but it apparently did not go too well.
>JavaScript is going to experience a much broader usage in the >future, predicting that was kind of obvious (specially since >gmail came to the show).
>
Everything is obvious in retrospect and à posteriori. And the hard part isnt predicting. The hard part is acting on the prediction and go against the status quo.
— MV
so, is there any hope for “thick” client-side apps?
they have their advantages:
better to work offline, can work better with the stuff we have stored on the machine, can access hardware, etc.
i would love “something” (call it a framework, if u want) like Eclipse RCP (forget about java, think about the stuff RCP provides you) but able to work seamlessly on and offline with the same code, able to store data online or offline, depending on finding a ‘net connection or not, interacting with hardware like a thick app but also able to run online like a webapp.
THAT would be write once, run everywhere.
>so, is there any hope for “thick” client-side apps?
>
Yes, as long as they are written in JS+CSS :-)
Actually that is what Adobe Air is.
And its no wonder that my/our proposal is called “Thin Server Architecture” ( http://www.thinserverarchitecture.com/ ).
>better to work offline, can work better with the stuff we
>have stored on the machine, can access hardware, etc.
>
Indeed. But that can be all be done in JS+CSS. The same way that you use Python or Java to do thick clients.
In fact its already being done: Adobe Air; Google Gears; browsers with HTML5; Firefox using XUL.
The full access to your machine hardware is just a matter of it being allowed by the OS to the interpreter/VM. Its currently disabled in browsers runtimes due to security issues.
— MV
well, i agree with you in all that.
the only thing keeping me from starting to use js in the thick client i need to do (need to use scanners, so it has to be thick, otherwise it would be a web app) is that im damn lazy. lol..
like i said, i want an RCP-like thing in js. maybe i should do it myself. lol
>like i said, i want an RCP-like thing in js.
http://280atlas.com/
— MV
great, thanks. :)
i knew about these guys but not about atlas.
too bad it’s in pay-to-play beta and mac-only… oh well…
Focusing on the template engine because it’s something I’m interested on, I did a _very basic_ PoC that illustrates my interpretation of Mario’s request. It’s available at http://github.com/cpinto/sample-jstemplate
>I did a _very basic_ PoC that illustrates my interpretation of >Mario’s request.
>
Yeah, sorta like that.
The use of LINKs in the HEAD is pretty ingenious but I would probably go with a single AJAX request with the required parameters and requested returns.
Or use the SRC attribute used in IFRAMEs to extend all HTML so that each one could have a data source (something like M$ DATASRC extension)
— MV
M$ (for a change) has it almost right:
Binding HTML Elements to Data
http://msdn.microsoft.com/en-us/library/ms531385%28VS.85%29.aspx
Replace DATASRC with SRC and DATAFLD with ID and its good to go.
— MV
And Rui Lopes ( http://twitter.com/ruidlopes ) has a good suggestion: Chain.js is pretty close to what’s needed for the templating system ( http://wiki.github.com/raid-ox/chain.js/ )
— MV
I get the src attribute, it allows for great granularity in terms of templating (eg. this div gets data from here, that div gets data from there) but I’m just not sold on adding non-standard attributes to tags.
As for chain.js, I found it lacking to be honest. Perhaps in the future it gets better.
Great post, appreciated your thoughts, and I certainly agree. Definitely will be linking to this. Hopefully Pintura will be a more complete realization of your aspirations, your ideas of pure JS frameworks has been one of inspirations behind it. For the time I am still expecting that devs will mostly be using native DBs for the storage until JS is a little faster, but I am including a very simple naive JS store for storing data in JSON files, and maybe expand/improve it with time.
“But *the* most important thing: it [the appserver] can not and must not allow the generation of HTML and the return of HTML to the browser/client!”
What will Google’s bots think about that?
>What will Google’s bots think about that?
>
Not good stuff I would think :-)
But this is an issue with the millions of pages out there that are using AJAX interactions.
I guess that this would be more of an issue with people who are too much worried about SEO. I’m not, I’m interested in the technical/architectural issues.
And eventually Google will index JS generated pages, like they started indexing Flash. Its just a matter of time.
— MV
Hey. I’m with you on the javascript vibe. I spent a good while developing a javascript orm, to run server side and client side, it was sickeningly awesome.
The problem for me was that js still isn’t supported very well on the server, it’s not easy to get it running on a decent enterprise grade server.
So I switched to using php on the server, until we get decent js interpreters built into our server products like lighttpd. I really want js on the server, but I really appreciate the ease of developing with a tool that has good debugging and a well used approach like php.
To be honest, I don’t think it’s a big compromise. I don’t do view control on the server, so that is done in javascript on the client anyway. Most of the work I do implementing any web app is still going to be reusable on the client side, and that’s where the beauty is in using the same language client & server side.
Hi,
We have been working on a stack very close to what you describe for a couple of years now. Including the most flexible client-side template system we could come up, called Live Markup. It is essentially Javascript extended with a bit of syntax handy for templating.
Also we are working on a serverside to match this. Currently we are a bit more XML focussed than JSON, but we’ll support both.
Check http://www.ajax.org and a bit of the live markup demos. We are still in beta so its not all polished yet.
My colleague Ruben wrote a short blog article about it,
http://www.rubendaniels.com/2009/10/22/live-markup-a-preview-of-a-hot-new-technology/
let me know what you think, i’m curious what you think of the templating.
Rik Arends
Hi Rik
Thanks for your presentation at Codebits :-)
>Including the most flexible client-side template system we
>could come up, called Live Markup. It is essentially Javascript
>extended with a bit of syntax handy for templating.
>
I assume that you are referring to the “a:page” “a:list” sort of markup. From the examples at the blog post I also see stuff like {} and foreach mixed in the HTML. I really do not like or agree with that approach.
We did that some 6 years ago and found out through experience that it was the wrong approach (check out http://web.archive.org/web/20030205211447/http://www.blogdot.org/ and http://web.archive.org/web/20020721153247/http://daaz.pt/ and View Source)
— MV
Hi Mario,
Why is it so important to have ID instead of CLASS as a mapping between the data and the template?
Those ID’s should disappear once they are used?
Otherwise, on iterations you would end up with multiple times the same ID’s
Richie Hindle (PyMeld’s author) even posted on the blog that he was seeing more value in being able to use any selector and not just ID.
Didn’t know you had a PhD on the subject ;)
Cheers,
Mic
Hi Mic
>Why is it so important to have ID instead of CLASS as a
>mapping between the data and the template?
>
Personal preference. I feel its the most adequate reference since by definition it should be unique, like any variable name. As such it makes sense for me to match up the uniqueness of variables or keys in the JSON hashes with the uniqueness of IDs
>Otherwise, on iterations you would end up with multiple
>times the same ID’s
>
One could generate IDs with a sequence number attached (id=”foo-X”). …
— MV
[…] teile die Meinung von Mario Valente, und erhoffe mir für die Zukunft performatere und schöner designte Webanwendungen auf dieser […]