Jono's technical notes

Scala traits to enhance existing java libraries

noreply@blogger.com (Jono) — Mon, 28 Sep 2009 07:18:00 +0000

Recently we moved some code over to Scala which uses Apache's httpClient, through laziness I used

method.getResponseBodyAsString. Apart from being poor form, it breaks our - nothing but errors in the Production logs rule at Spiral Arm.

To alleviate the coding smell we created a trait to read the body as a stream instead of a string. Rather a simple task. The nice things about the trait are the use of the self reference.

self:HttpMethod =>

This means 'I expect to be mixed with something of this type' nicely reducing the scope where the trait can be applied and is also self documenting. We could have returned a None Option instead of an empty String, meh. It's an implementation issue and not a huge issue.

trait SafeResponseBody {

self:HttpMethod =>

def getResponseBodyUpTo(to:Int):String = {

val in = new InputStreamReader(this.getResponseBodyAsStream, "UTF-8")

val buffer = new Array[Char](to)

val n = in.read(buffer,0,to)

n match {

case -1 => ""

case 0 => ""

case _ => new String(buffer, 0, n)

}

Using the trait is nice and clean, when creating an instance of the particular HttpMethod we mix in our trait and then have access to it's methods.

val method = new PostMethod(url) with SafeResponseBody

...

method.getResponseBodyUpTo(160)

Scala frameworks

noreply@blogger.com (Jono) — Wed, 18 Mar 2009 05:31:00 +0000

I'm currently trying to get my head around Lift, but at the moment it seems about 10 perspiration 90% magic, It is incredible concise, which is wonderful after spending the last few days bashing my head against the struts wall.

I defaulted to looking at lift as I thought it was the only Scala web framework available. Today I spent a bit of time looking round for any others and below are some brief notes on the frameworks found.

Lift

Lift is the most popular of the Scala based framework. It is built round the notions of convention over configuration and separation of concerns - view are XHTML with lift tags, there can be no intermingling of presentation and code in the views. It has recently gone to version 1.0 and has an active community willing to help. It is easy to get started as lift requires maven and nothing else to create a project.

Sweet

Sweet is another convention over configuration framework with a grails like convention over configuration command line utility to quickly generate controllers. Sweet also uses maven to manage dependencies. Sees more flexible than lift, you are not bound to a session based implementation. This comes at the price that you will need to do more of the work yourself.
It is more standard MVC framework, but in scala, than a complete change that you get with lift. Site site seems to have a friendly tone and is some what more accessibly than the lift site. It is early days with the current version at 0.0.2.642 and 4 developers, it does however have a semi-active community.

Web flavor

flavor appears to be a scala replacement for php/cgi using scala script in the page, it looks like it also supports templating similar to lift. flavor currently has one developer and
I couldn't see any sign of a community, to be fair I didn't really look very hard.

Pinky

To quote Pinky's home page "Pinky is a simple Scala REST/MVC web framework built on top of Guice Servlet." It supports XML/Json generation out of the box no configuration nor batteries required. It appears to be about 7 weeks old and have a single developer and some what unsurprisingly no community that I could find.

Have I missed any others, do I have any wildly incorrect facts ?

State of Scala in the IDE

noreply@blogger.com (Jono) — Thu, 19 Feb 2009 09:29:00 +0000

So I've been playing with Scala over the last few days and trying to get my head back in functional land, which means at some stage Im either going to go postal or start talking backwards. I used vi for the first few days until I couldn't handle being with out ctrl + space any longer.

I've not tried Intellij, The subversion interface disturbs me greatly, there is a review of the Scala integration below. For me unless intellij pulls something phnomenal out of the bad it is really between eclipse and netbeans.

Netbeans

I've only had a quick play with the latest plugin for 7.0 but it seems to handle refactoring, indentation, running and code completion fine, the site seems to
insinuate there is debugging, but buggered if I could find it, perhaps a less than cursory glance may have helped.

eclipse

I've been using eclipse for a few days now, and apart from a few hick-ups getting eclipse to compile or recognise the Scala classes it has been pretty
good. Running and debugging code is fine, code completion works, indentation is some weird sort of help that I have failed to under stand but anyone
with a post grad degree will be able to explain to me. The one thing it is missing and ones again causes me tears is it's lack of refactoring.

Both support maven so switching between IDE's is trivial, which is nice.

scala intellij integration

How to install trac 0.11 on OS X ( 10.4.10)

noreply@blogger.com (Jono) — Thu, 11 Oct 2007 10:39:00 +0000

The following are notes on how to install and configure trac on OS X with multiple projects with a dedicated user to maintain the trac installation. These notes have been derived from several sources and are primarily so I can reproduce this at a later date without thinking, they are, unsurprisingly

http://trac.edgewall.org/
http://modpython.org/

You'll need the following

sudo / root access
internet access
python 2.3.5
sqlite3

You will need to download

Apache 2.2.4
Mod_python 3.3.1
setuptools

Install Apache 2.2.4

Follow Richard's instructions, see step 6 for an overview

./configure --prefix=/Users/jono/Applications/apache/2.2.4 --enable-mods-shared=most --enable-ssl --with-mpm=worker --without-berkeley-db --enable-proxy  

make
sudo make install
cd ~/apache/2.2.4/bin
sudo ./apachectrl start

check there is something sane here http://127.0.0.1/

Install mod_python 3.3.1
- configure and install
```
./configure --with-apxs=/Users/jono/Applications/apache/2.2.4/bin/apxs
make
sudo  make install
```
- tell apache about it
  add
```
LoadModule python_module /Users/jono/Applications/apache/2.2.4/modules/mod_python.so
```
  to
```
/Users/jono/Applications/apache/2.2.4/conf/httpd.conf
```
- test apache is talking to python
  1. edit httpd.conf add
    
    <Location /mpinfo>
    SetHandler mod_python
    PythonHandler mod_python.testhandler
    </Location>
  2. restart apache and see what we get
```
cd ~/apache/2.2.4/bin
sudo ./apachectrl
http://127.0.0.1/mpinfo
```
    you should get a table showing various System information about apache, python and the request headers, including who the process is running as. In our case it is daemon - make a note of this you'll need it later.
  3. clean up
    
    remove the following from httpd.conf
    <Location /mpinfo>
    SetHandler mod_python
    PythonHandler mod_python.testhandler
    </Location>
Install python modules required by the latest version of trac ( 0.11)
- install setuptools
  1. this will need to be done as root so you have write permissions to /System/Library/Frameworks/Python.framework/Versions/2.3/bin
    sudo python ez_setup.py
  2. link it to a bin directory in your path - This should be a single line!
    sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.3/bin/easy_install easy_install
  3. test it, type eas and you should get tab completion, well, assuming you are using bash.
  4. install Genshi
    easy_install Genshi
  5. install pysqlite
    - make sure easy_install is going to install the correct version,it should be 2.3.2 or above
      easy_install -n pysqlite
    - if the above seems okay, install it
      easy_install pysqlite

Install trac
This is some what more convoluted than it needs to be, once edgewall release a complete version (0.11) of trac, this will be

easy_install trac
However at present this installs 10.4, which is not what we want, so it is worth trying easy_install -n trac to see if it is going to install 0.11, until then, there is :
- check out the latest code from svn, in my case this was revision 6049.
  svn co http://svn.edgewall.org/repos/trac/trunk trac
- install trac
  sudo python ./setup.py install
- link the trac binaries to a bin directory in your path.
  sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.3/bin/ trac-admin trac-admin
  sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.3/bin/ tracd tracd
- If you want to be sure trac is installed and working before continuing ( probably a good idea ) check the Observations notes below.
set up trac user and environment
- create a trac user - see my previous post
- add them to the daemon group - or whatever user apache is running at
  sudo nicl / -merge /groups/daemon users daemon
- as the trac user create the projects and set the correct permissions
  1. create projects
    trac-admin project00 initenv
    trac-admin project01 initenv
  2. set the permissions on the trac projects
    chown -R :daemon project00
    chown -R :daemon project01
    chmod -R g+rw project00
    chmod -R g+rw project01
Configure apache to talk to trac via mod_python
- add the following to httpd.conf
  <Location /trac> SetHandler mod_python PythonInterpreter main_interpreter PythonHandler trac.web.modpython_frontend PythonOption TracEnvParentDir /Users/trac PythonOption TracUriRoot /trac </Location><LocationMatch "/trac/[^/]+/login">AuthType Basic AuthName "Trac" AuthUserFile /Users/trac/.htpasswd Require valid-user </LocationMatch>
- as the trac user populate .htpasswd
  htpasswd -c /Users/trac/.htpasswd jono
- as the trac user configure trac permissions
  1. add an administrative user for the projects
    trac-admin /Users/trac/project00 permission add jono TRAC_ADMIN
    trac-admin /Users/trac/project01 permission add jono TRAC_ADMIN
  2. remove anonymous and authenticated permissions from all projects, this is needed so users can not log into projects others than there own and is probably just a good idea ;)
    trac-admin /Users/trac/project00 permission remove anonymous '*'
    trac-admin /Users/trac/project00 permission remove authenticated '*'
    trac-admin /Users/trac/project11 permission remove anonymous '*'
    trac-admin /Users/trac/project11 permission remove authenticated '*'
  3. bounce apache
    sudo ./apachectl stop
    sudo ./apachectl start
  4. try and login
    http://127.0.0.1/trac/project0
    http://127.0.0.1/trac/project1
restrict access to projects - currently every entry into .htpasswd has access to all projects, as an authenicated user, which would allow users from one project to browse and edit the wiki or another - not an ideal situation, so we do the following
- remove ability to see project listing - what people can't see they can't try and access
  1. update httpd.conf - change Locaiton from <Location /trac> to <LocationMatch "/.+/"> remembering to also update the closing Location tag to LocationMatch
  2. bounce apache
    sudo ./apachectl stop
    sudo ./apachectl start

Finally

Things to do:

get svn access working
consider https
consider using digest rather than basic athenication
virtual hosts

Observations

install using the default locations for python, installing with user specific locations will cause a great deal of pain, if you know what you are doing with PYTHONPATH, you will probably be okay, then again, you probably aren't going to be reading this either :)

if you ignored me and tried installing setuptools without root access you will have noticed the binary failed to install and now you are wondering how to get it installed.
sudo python ez_setup.py -U setuptools

restarting apache,there are notes on either the trac or mod_python site about this, it would appear that sudo ./apachectl restartdoesn’t always work, this is usually when you are tired and it has been working all day, save yourself pain just use.
sudo ./apachectl stop
sudo ./apachectl start
if trac isn’t playing you may want to make sure it is installed correctly, try the following
1. create a dummy project
  trac-admin testTrac initenv
2. start the trac daemon
  tracd --port 8000 /Users/jono/Documents/trac/testTrac
3. point a browser to
  http://localhost:8000/testTrac
4. if it all looks okay, you can delete this now

AIMIA Launches Mobile Advertising Guidelines

noreply@blogger.com (Jono) — Mon, 03 Sep 2007 11:20:00 +0000

I attended the "The Future of mobile Advertising" presented by the AIMIA on Tuesday morning, this was mainly a chance to launch the AIMIA mobile advertising guidelines, There was little earth shattering information and most of it is common sense and a great deal of "use us, other people don't know what they are talking about".

Disappointments

Mobile advertising is going to be amazing, this year 6-7 million, 200 Million in 3 years, 1.3 Billion by 2010, 10 - 20% of all advertising, little of this seemed based on fact, seeing as the numbers were so wildly different. The six to seven million was actually the previous 12 months, I believe, otherwise it was YTD.
People are trying to work out how to measure mobile advertising, for some reason they believe this isn't going to be click through or SMS's sent or impression based.
Content, including advertising - such as banner adverts need to be tailored to the individual device, no great surprises there, scrolling to see adverts either horizontally or vertically is bad.
Attitude of some of the presenters towards technical folk rather than advertisers, referring to developers as "techno dorks" and the unless companies using a developer who knew XHTML would most likely get it wrong. Yes, the guy got on my tits - I guess they have highly trained advertisers in that particular company creating mobile sites and definitely NO techno dorks lowering the tone.

Possibly useful information.

from Hubert Kjellberg, Head of Media ,Ericcson.

15 - 39 Demographic are leaving print media or not even bothering with it.
Consuming all media in the same way as the internet media.
Carriers are moving away from walled gardens - huzzah.
Figures for passive television consumption are dropping, still healthy, but dropping.
A group of 14 - 15 year olds were interviewed about media consumption and there preferences were.

internet
mobile
ipod
television

When developers of mobile sites need to be aware of local regulations, these can be through various bodies, broadcasting regulator, AIMIA, etc.

from someone at publicis digital

mobile phones are the 3rd screen,
19 million phones in AU, not bad for a population of just over 21 million,
4.5 million 3G phones, possible tipping point ?
Terms

on-deck --> teleco site/portal
off-deck --> content publishers

local knowledge is important in mobile advertising ( one would assume mobile content full stop ), as usage varies around the world. One wonders if this is culturally, development of mobiles services or possibly the type of devices available in the market.

someone from communicator.com.au

click through for off deck is ~ 3 - 6 % for mobile advertising campaigns,
consumer perception that data is expensive, this is a barrier to adoption, consumers need to be educated,
data charges are falling,
age of users is moving up, 25 - 40s watching mobile videos,
real time behavioral targeting will be important

some at tiger spike

most traffic is on portal,
in the next 6 -12 months we will see more off portal traffic,
Need to follow the 1,2,3 rule :

One point of entry,
No more than 2 seconds to load,
No more than 3 clicks to get anywhere - 30% lose per click, so not much point.

Mobile sites need to be live sites if they are going to be around for more than a month, so have content to put up, this will also get people interested in coming back, consider a news link, then people will know the site is alive,
watch for bots mimicking mobile devices, they can show false spikes in traffic,
things that companies are doing wrong at present :

Not considering company strategy,
being to clever, billboard that has a web site where you can download an app for your phone that will let you read a bar code,
copying web site to be a mobile site,
not getting the right advice.

someone from vodafone mobile advertising

a new group set up to provider agencies the ability to advertise on the vodafone LIve! portal, rather than just seeing vodafone adverts,
starting in AU, being pushed out worldwide,
vodafone talking with other carriers about cross carrier advertising and other possibilities - early days and cross carrier agreements move at ( pre global warming ) glacial speed.

someone from Sensis

Example of a mobile banner advert for KFC that linked through to a site that used carrier data to build a map showing you where the closest KFC was based on your location, also had a form so you could enter your mobile number and they would SMS you a voucher for a free or discounted burger.

Creating a user from the commandline on OS X

noreply@blogger.com (Jono) — Tue, 13 Mar 2007 06:37:00 +0000

The following will create a user USER_NAME in the group USER_NAME
with a home directory /Users/USER_NAME, it assumes you have sudo
access and are currently in /Users

Find the next free uid by running the following
create user, assuming a free uid of 505 from the previous step.
Set password
create group
create user directory
change ownership
make sure you can login

As an aside here is how to delete a user via the command line.

check the user exists
delete the users group
delete the user
delete user home directory

Taken from

http://www.mcelhearn.com/article.php?story=2004110211244242
http://mikesalsbury.com/mambo/content/view/130/2/
http://www.oreillynet.com/mac/blog/2006/04/deleting_mac_os_x_users_remote.html
http://developer.apple.com/documentation/Darwin/Reference/ManPages/index.html
http://docs.info.apple.com/article.html?artnum=60038

Switch

noreply@blogger.com (Jono) — Thu, 07 Dec 2006 10:28:00 +0000

Not from Windows to Apple, from Apple to Apple - It has been two 1/4 years since I switched from windows / Linux. Now I've moved from my trusty 15" PowerBook G4 to a brand spanking new 17" MacBookPro, my god it is fast and the extra real estate is most welcome.

My only complaint would be the hard disk speed /size options, Its either small and fast or big and slow - I went small and fast 7200 RPM but only a 100 GB, after moving data I only have 30 GB free and it is less than a month old !

I can see me making rather a lot of use of my external drive.

Port redirecting

noreply@blogger.com (Jono) — Sat, 02 Dec 2006 11:24:00 +0000

This is probably obvious to everyone but me, however I'll put it here in case there is another who finds it helpful.

The following redirects requests from 80 to 8080, meaning you don't have to run your
webapp as root or have annoying urls such as http://localhost:1234/myapp.

Jono@moose:xinetd.d $pwd
/etc/xinetd.d
Jono@moose:xinetd.d $cat redirect
service www
{
socket_type = stream
wait = no
nice = 10
user = root
server = /usr/bin/nc
server_args = -w 3 localhost 8080
instances = 40

}

Code for the stupidest member of the team

noreply@blogger.com (Jono) — Thu, 25 Aug 2005 09:53:00 +0000

While we would all like to work with the Fowlers and Blochs of the world, there are not enough of them to go around. Which means either changing career, or be accepting that there are less talented people in your industry, and by a cruel twist of fate you may well work with one of them.
With this in mind, what should one do ? The first thing is to not instantly assume people will understand what is going on from your code. Some documentation, is a good, thing, even if it is within the code itself.
Better yet tests your fellow developers can run, unit and integration tests are ideal for this, they not only show that the code works - they show HOW it works.
Secondly try not to use the most esoteric aspects of the language, they will only confuse and befuddle your coworkers, the most readable solution is the best and usually the simplest.
This will always be relative, the stupidest member of a team writing control code for a plane or power station is most likely to be _less_ stupid that the brightest member of a website.