mvalente.eu/mv.* Heavy Mental Stuff

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Serverside Javascript Tracemonkey Performance – Part V
Date: Wed, 27 Aug 08 01:50:21 GMT

Finally had some time free to run the Sunspider tests on
serverside Javascript 1.6 and JS1.8tracemonkey. The results are
listed below.

http://svn.webkit.org/repository/webkit/trunk/SunSpider/tests/

Performance wise there’s not much of notice. Results are
basically in line with Brendan Eich’s results for browserside
tests.

http://weblogs.mozillazine.org/roadmap/archives/2008/08/tracemonkey_javascript_lightsp.html

One thing is of notice though: JS1.8 throws a segfault in
at least 5 of the tests. I dont have much time to debug right
now, but a strace shows that the last system call was a call
to gettimeofday, after a bunch of read()s containing js source
code lines. Maybe a call to date functions… or the fact that
I patched and compiled in the not-recomended jsfile.c File object
support ;-)… Will have to look into it later…

—————————————
3d-cube.js
JS 1.6
real 0.59
user 0.55
sys 0.01

JS 1.8
real 0.08
user 0.06
sys 0.01
—————————————
3d-morph.js
JS 1.6
real 0.69
user 0.61
sys 0.04

JS 1.8
real 0.06
user 0.04
sys 0.02
—————————————
3d-raytrace.js
JS 1.6
real 0.47
user 0.41
sys 0.02

JS 1.8
./perftest: line 1: 13589 Segmentation fault js18 -j $f
real 0.01
user 0.01
sys 0.00
—————————————
access-binary-trees.js
JS 1.6
real 0.27
user 0.22
sys 0.03

JS 1.8
real 0.10
user 0.08
sys 0.01
—————————————
access-fannkuch.js
JS 1.6
real 0.97
user 0.93
sys 0.00

JS 1.8
./perftest: line 1: 13593 Segmentation fault js18 -j $f
real 0.03
user 0.02
sys 0.00
—————————————
access-nbody.js
JS 1.6
real 0.60
user 0.44
sys 0.06

JS 1.8
real 0.07
user 0.05
sys 0.01
—————————————
access-nsieve.js
JS 1.6
real 0.56
user 0.50
sys 0.00

JS 1.8
real 0.03
user 0.03
sys 0.01
—————————————
bitops-3bit-bits-in-byte.js
JS 1.6
real 0.26
user 0.26
sys 0.00

JS 1.8
real 0.00
user 0.00
sys 0.00
—————————————
bitops-bits-in-byte.js
JS 1.6
real 0.40
user 0.38
sys 0.00

JS 1.8
real 0.01
user 0.01
sys 0.00
—————————————
bitops-bitwise-and.js
JS 1.6
real 0.34
user 0.29
sys 0.00

JS 1.8
real 0.03
user 0.01
sys 0.01
—————————————
bitops-nsieve-bits.js
JS 1.6
real 0.66
user 0.56
sys 0.00

JS 1.8
real 0.04
user 0.00
sys 0.00
—————————————
controlflow-recursive.js
JS 1.6
real 0.14
user 0.12
sys 0.00

JS 1.8
real 0.07
user 0.07
sys 0.00
—————————————
crypto-aes.js
JS 1.6
real 0.46
user 0.32
sys 0.04

JS 1.8
./perftest: line 1: 13609 Segmentation fault js18 -j $f
real 0.01
user 0.00
sys 0.00
—————————————
crypto-md5.js
JS 1.6
real 0.38
user 0.27
sys 0.01

JS 1.8
./perftest: line 1: 13611 Segmentation fault js18 -j $f
real 0.02
user 0.02
sys 0.00
—————————————
crypto-sha1.js
JS 1.6
real 0.35
user 0.25
sys 0.01

JS 1.8
real 0.02
user 0.02
sys 0.00
—————————————
date-format-tofte.js
JS 1.6
real 1.03
user 0.69
sys 0.09

JS 1.8
./perftest: line 1: 13615 Segmentation fault js18 -j $f
real 0.08
user 0.07
sys 0.00
—————————————
date-format-xparb.js
JS 1.6
real 1.74
user 1.37
sys 0.17

JS 1.8
real 0.19
user 0.15
sys 0.02
—————————————
math-cordic.js
JS 1.6
real 0.63
user 0.51
sys 0.00

JS 1.8
real 0.05
user 0.04
sys 0.00
—————————————
math-partial-sums.js
JS 1.6
real 0.42
user 0.36
sys 0.02

JS 1.8
real 0.03
user 0.03
sys 0.00
—————————————
math-spectral-norm.js
JS 1.6
real 0.28
user 0.21
sys 0.04

JS 1.8
real 0.02
user 0.01
sys 0.00
—————————————
regexp-dna.js
JS 1.6
real 0.60
user 0.50
sys 0.05

JS 1.8
real 0.37
user 0.33
sys 0.01
—————————————
string-base64.js
JS 1.6
real 1.10
user 0.94
sys 0.07

JS 1.8
real 0.03
user 0.01
sys 0.02
—————————————
string-fasta.js
JS 1.6
real 0.77
user 0.66
sys 0.05

JS 1.8
real 0.23
user 0.20
sys 0.02
—————————————
string-tagcloud.js
JS 1.6
real 0.63
user 0.58
sys 0.00

JS 1.8
real 0.23
user 0.19
sys 0.02
—————————————
string-unpack-code.js
JS 1.6
real 0.54
user 0.48
sys 0.01

JS 1.8
real 0.33
user 0.28
sys 0.04
—————————————
string-validate-input.js
JS 1.6
real 0.43
user 0.35
sys 0.04

JS 1.8
real 0.10
user 0.07
sys 0.02

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Serverside Javascript Tracemonkey Performance – Part IV
Date: Mon, 25 Aug 08 02:06:21 GMT

Before starting in on the Dromaeo and Sunspider tests, I decided
to run some tests to compare JS1.8tracemonkey with other languages.

For C, I created a couple of tests similar to Mike Shaver’s simple loop
but using instead the loop-sum and the function-sum tests from Sunspider
ubench tests.

http://shaver.off.net/diary/2008/08/22/the-birth-of-a-faster-monkey/

As you can see from the output below, JS1.8tracemonkey only comes close
to unoptimized gcc in a simple for loop. If the loop contains some
statements, as is the case, Tracemonkey is slower 2x/3x than compiled C.

To compare with other languages, I also created a direct copy of Tim Bray’s
Ruby code for his Wide Finder project, but with dismal results. My script clocks
in at nearly 5 secs using his o10k.ap 2Mb file sample (which would mean >500secs
with the o1000k.ap 200Mb file).

http://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder
http://mv.asterisco.pt/Projects/wf1.js

It seems that the only language missing an entry in the Wide Finder challenge
is Javascript. I believe that the only reason for that is that people
just arent used to use Javascript to perform these type of tasks, due to the
misconception that “its for the browser and you cant access files”. I’m
curious to find out if with JS1.8tracemonkey we can get similar performance
to the other languages/contestants…

Tomorrow I hope to try and reproduce Santiago Gala’s and Fredrik Lundh
techniques and have a script that is more in line with the general results
for other languagues (meaning a ballpark 5-8 seconds for the 200Mb sample
or 50-80ms for the 2Mb sample). Or someone more versed in generators,
iterators, map/reduce and all that shiny new stuff might be kind enough
to send in the script :-)…

http://effbot.org/zone/wide-finder.htm
http://www.tbray.org/ongoing/When/200x/2007/10/30/WF-Results

—————————————-

[root@localhost perfjsvsc]# cat loop-sum.js; js18 -j loop-sum.js; time -p js18 -j loop-sum.js
var count = 99999999;
var sum = 0;
for (var i = 0; i < count; i++) {
sum = i + count;
}

real 0.84
user 0.19
sys 0.00

—-

[root@localhost perfjsvsc]# cat loop-sum.c; gcc -o loop-sum loop-sum.c; ./loop-sum; time -p ./loop-sum
int count = 99999999;
int i, sum = 0;

main()
{
for (i = 0; i < count; i++)
{
sum = i + count;
}
}
real 0.39
user 0.09
sys 0.00

—-

[root@localhost perfjsvsc]# cat function-sum.js; js18 -j function-sum.js; time -p js18 -j function-sum.js
function f(x, y, z)
{
return x + y + z;
}

for (var i = 0; i < 99999999; ++i)
f(1, 2, 3);

real 1.76
user 0.41
sys 0.00

—-

[root@localhost perfjsvsc]# cat function-sum.c; gcc -o function-sum function-sum.c; ./function-sum; time -p ./function-sum
int f(int x, int y, int z)
{
return x + y + z;
}

main()
{
int i;

for (i = 0; i < 99999999; ++i)
f(1, 2, 3);
}

real 0.69
user 0.17
sys 0.00

—————————————-

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Serverside Javascript Tracemonkey Performance – Part III
Date: Sun, 24 Aug 08 19:01:21 GMT

As expected the Sunspider ubench Javascript performance tests
on serverside JS 1.6 and JS 1.8 (tracemonkey) run much faster
when on a halfway decent machine. Below you can find the tests
when running on a Xen VM (underlying hardware is some kind of
quad-core machine).

In this case JS1.6 now takes 9.940ms to complete all tests
(a small decrease from the 11s on the UML VM and about double
the 4.1982ms of the browserside testes) while JS1.8tracemonkey
now takes 0.300ms (halving the previous serverside tests at the
UML VM and coming close to the 185ms on the browserside tests.
Huge difference, now around a +-30X increase!

If someone wants to and can run these tests in a bare metal
Linux machine and needs the files, let me know. The browserside
tests made by Brendan Eich dont mention the hardware used to run
the tests but I would like to see some serverside tests in a
decent machine which I’m sure would beat the browserside times.

[root@localhost ubench]# cat ubench.out
—————————————
function-closure.js
JS 1.6

real 0m0.399s
user 0m0.340s
sys 0m0.030s

JS 1.8

real 0m0.011s
user 0m0.000s
sys 0m0.010s
—————————————
function-empty.js
JS 1.6

real 0m1.869s
user 0m1.760s
sys 0m0.000s

JS 1.8

real 0m0.065s
user 0m0.040s
sys 0m0.000s
—————————————
function-missing-args.js
JS 1.6

real 0m1.609s
user 0m1.600s
sys 0m0.000s

JS 1.8

real 0m0.047s
user 0m0.040s
sys 0m0.000s
—————————————
function-sum.js
JS 1.6

real 0m1.769s
user 0m1.620s
sys 0m0.000s

JS 1.8

real 0m0.048s
user 0m0.040s
sys 0m0.000s
—————————————
loop-empty-resolve.js
JS 1.6

real 0m0.392s
user 0m0.350s
sys 0m0.000s

JS 1.8

real 0m0.027s
user 0m0.020s
sys 0m0.000s
—————————————
loop-empty.js
JS 1.6

real 0m1.865s
user 0m1.820s
sys 0m0.000s

JS 1.8

real 0m0.050s
user 0m0.040s
sys 0m0.000s
—————————————
loop-sum.js
JS 1.6

real 0m2.027s
user 0m1.950s
sys 0m0.010s

JS 1.8

real 0m0.052s
user 0m0.050s
sys 0m0.000s

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Serverside Javascript Tracemonkey Performance – Part II
Date: Sun, 24 Aug 08 18:01:21 GMT

I just ran the Sunspider ubench Javascript performance tests
on serverside JS 1.6 and JS 1.8 (tracemonkey).

http://svn.webkit.org/repository/webkit/trunk/SunSpider/tests/ubench/

The results are listed below. Summarizing, JS1.6 takes 11.408ms to
complete all tests while JS1.8tracemonkey takes 0.705ms. Huge difference,
around the ballpark +-20X increase seen in browserside tests. Keep in
mind that the results below were performed in a usermodelinux virtual
machine, running within a Xen VM, and as such performance is worse than
the browserside tests available on the web (ex. John Resig’s site)

[root@localhost ubench]# less ubench.out
—————————————
function-closure.js
JS 1.6

real 0m0.799s
user 0m0.080s
sys 0m0.010s

JS 1.8

real 0m0.066s
user 0m0.000s
sys 0m0.000s
—————————————
function-empty.js
JS 1.6

real 0m2.008s
user 0m0.500s
sys 0m0.000s

JS 1.8

real 0m0.122s
user 0m0.010s
sys 0m0.000s
—————————————
function-missing-args.js
JS 1.6

real 0m2.019s
user 0m0.470s
sys 0m0.000s

JS 1.8

real 0m0.109s
user 0m0.010s
sys 0m0.000s
—————————————
function-sum.js
JS 1.6

real 0m1.896s
user 0m0.450s
sys 0m0.000s

JS 1.8

real 0m0.112s
user 0m0.000s
sys 0m0.000s
—————————————
loop-empty-resolve.js
JS 1.6

real 0m0.494s
user 0m0.100s
sys 0m0.000s

JS 1.8

real 0m0.085s
user 0m0.000s
sys 0m0.000s
—————————————
loop-empty.js
JS 1.6

real 0m2.000s
user 0m0.520s
sys 0m0.000s

JS 1.8

real 0m0.102s
user 0m0.010s
sys 0m0.000s
—————————————
loop-sum.js
JS 1.6

real 0m2.192s
user 0m0.520s
sys 0m0.010s

JS 1.8

real 0m0.109s
user 0m0.010s
sys 0m0.000s

— MV

Path: mv.asterisco.pt!mvalente
From: mvale…@ruido-visual.pt (Mario Valente)
Newsgroups: mv
Subject: Serverside Javascript Tracemonkey Performance – Part I
Date: Sun, 24 Aug 08 15:11:21 GMT

Last night I finally got the 1.8 release of the
C Spidermonkey Javascript interpreter/compiler/VM to
run serverside (the so-called Tracemonkey version).

I intend to run some performance tests between JS 1.6
and JS 1.8 (tracemonkey), namely using the Sunspider and
Dromaeo benchmarks. Not an easy task since they are setup
for browser side testing, but I’ll get there.

Meanwhile, here’s the performance with the “perfect.js”
test that comes bundled in Spidermonkey:

———————————————————
[root@localhost src]# time js perfect.js

A number is ‘perfect’ if it is equal to the sum of its
divisors (excluding itself).

The perfect numbers up to 50000 are:
6 = 1 + 2 + 3
28 = 1 + 2 + 4 + 7 + 14
496 = 1 + 2 + 4 + 8 + 16 + 31 + 62 + 124 + 248
8128 = 1 + 2 + 4 + 8 + 16 + 32 + 64 + 127 + 254 + 508 + 1016 + 2032 + 4064
That’s all.

real 0m4.053s
user 0m0.720s
sys 0m0.020s
———————————————————

———————————————————
[root@localhost src]# time js18 -j perfect.js

A number is ‘perfect’ if it is equal to the sum of its
divisors (excluding itself).

The perfect numbers up to 50000 are:
6 = 1 + 2 + 3
28 = 1 + 2 + 4 + 7 + 14
496 = 1 + 2 + 4 + 8 + 16 + 31 + 62 + 124 + 248
8128 = 1 + 2 + 4 + 8 + 16 + 32 + 64 + 127 + 254 + 508 + 1016 + 2032 + 4064
That’s all.

real 0m3.622s
user 0m0.570s
sys 0m0.010s
———————————————————

— MV