A tinyBenchmark

One of the first things I was shown how to do in a Smalltalk Image was a tinyBenchmark. It is a particular way to get a measure of the performance of Squeak and Pharo System. I thought it would be interesting to compare the results of that between different Smalltalk Systems. I’ll take those that I have most heard of: Pharo 1.2, which uses the COG VM , Squeak 4.2, GNU Smalltalk 3.2.3, VA Smalltalk 8.0.3 and VisualWorks 7.7.1.

However, tinyBenchmarks is not available on every Smalltalk being checked. So we’ll standardize the benchmarking code in every image to be the same (in Integer class, methods tinyBenchmark, benchmark and benchFib). The experiment will be repeated 10 times with each machine to reduce error imposed by GC and OS via issuing
(1 to:10) do: [:e | Transcript show: 1 tinyBenchmarks; cr.]. . All the experiments were run in an Acer Aspire 4540 Notebook with Ubuntu Linux 10.10 32-bit. (AMD Athlon II x2 M300 (2GHz, 1MB L2 Cache), with 2GB RAM).

So… here are the results!
Bytecodes

Message Sends

My impression is that, if you are working on some non commercial effort, you should go for VisualWorks ASAP. If you want to sell sth made on Smalltalk, Pharo is really the best option right now.

I hope this comparison is useful for somebody!

UPDATE There was some discussion about this post on the official Pharo mailing list. I will work on redoing this experiments during the next days, but for now, here are some extra facts that could be useful for the analysis of the results previously shown:

  • The Pharo 1.2 benchmarks are run with the following Cog VM: ‘SimpleStackBasedCogit VMMaker-oscog.34 uuid: ec174a95-5f42-4fe0-88f0-4baff13dd3ce Oct 4 2010’. The method to get this information is Smalltalk getSystemAttribute: 1008 (Thanks Elliot!). In Squeak 4.2, this returns nil. The Smalltalk vmVersion returns ‘Squeak4.1 of 17 April 2010 [latest update: #9957]’.
  • Several people have pointed out the comparison between Squeak and Pharo as unfair, since the COG VM is mostly responsible for the performance difference, and it is also available for Squeak. However, the official distribution of Squeak does NOT still come with the COG VM, but the Squeak VM. Using that VM you get the results previously shown.

    I repeated the experiment using the same VM (the COG that came with Pharo 1.2) and the Squeak Image. With the same VM, the results are very similar in performance between Squeak and Pharo. However, Pharo has, in average, 0,14% more bytecodes and 3.89% more message sends/second.

  • Anuncios

TR: Adapting Scrum to Managing a Research Group

Technical Report by Mchael Hicks and Jeffrey S. Foster. Dept. of Computer Science University of Maryland, College Park. Available at http://www.cs.umd.edu/~mwh/papers/score.pdf

Introduces SCORE, a research methodology based on Scrum and Agile Methodologies. The key factor is a three-times-a-week 15 minute meeting with the entire research group where everybody speaks about their improvements. After this, individual in-depth meetings are scheduled. Also, it includes an extra reading group where people meet to comment a particular paper that is assigned for reading each week. This might be interesting to do!

Installing Newspeak Starter Pack in Fedora 14 x86_64

This week I changed my Linux OS from Ubuntu 32 bit to Fedora 64 bits. Also, I’m starting to work with the Newspeak Language(more info at newspeaklanguage.org), so I decided to install the Starter Pack available in the website for Linux.

However, the – still based in Squeak Smalltalk- Virtual Machine only runs in 32 bits, so when you try tu use it, it complies that things are missing (Abstract for Pros: Download the starter pack and install the required 32 bit libraries. stop reading this.)

You should run the following command:
# yum install libuuid.i686 libXext.i686 libSM.i686

after this. just run the vm. I copied the nsvm and SqueakV39.sources files to /usr/local/bin, so you can run directly and everywhere you have an image a command like
$ nsvm ns101.image
and, after this, start using Newspeak!