Find Mendeley users

I haven’t written anything here for 3 weeks and a bit… but not because I didn’t do anything – quite the opposite, I finished or almost finished some projects.

First small project: a simple Web page so Mendeley users can add themselves on a map: find Mendeley users (we haven’t publicized it from Mendeley so there are almost no users, but if you have a Mendeley account it’s easy to add yourself: sign in and move the marker).

I started it “just for fun” at home, then See Wah and I kept working on it during a hackday at Mendeley. See Wah then kept working on it on another hackday for the UI/UX (User Interface and User Experience) (I’m not good designing UI and UX). And on the last hackday and a few more changes after that the map is live!

This is the first time that I’ve released something that uses MongoDB (it fits the data model that I needed really well). And for hosting I use Heroku… I really liked to used Heroku for the first time, and it was really easy to implement.

So: find Mendeley users

Happy 2015!

I hope everyone have a good 2015!

And for the IT colleagues, and as I said when I started my holidays: I wish a bug free and feature rich year! :-)

Recent change in the blog theme

A few weeks ago I changed this blog’s theme. Since 2006 it had had the same theme, which was getting a bit old, it had fixed width, and it wasn’t using the latest WordPress improvements like it didn’t have support for widgets.

I had a few requirements for the new theme: I wanted a 2 or 3 columns (I think that I need a sidebar for some links and information), I wanted to be able to have the header image since it has a long story behind it (it’s a painting of one of the original authors of the blog), I didn’t want to waste too much space with the heading, I wanted the same colour scheme, I wanted to be responsive. I also wanted a clear typography.

I’m surprised about how good WordPress themes work: there are many to choose from, they all work in all versions of WordPress. I found a few that look “almost like I wanted” but they had some problem, usually they had only one column or the header wasn’t as I wanted (I wanted to re-use the heading that I had, and responsive themes like to have the heading and then the text underneath instead of on it).

Anyway, I finally decided to use Atahualpa. It’s very customizable. I made the heading smaller and instead of the way that used to be I moved the text next to the image… it was the only way that I could make it all fit. It seems that Atahualpa even if it’s a modern theme I changed a bit to a retro looking, in the good way of retro. After a few weeks I can say that I like it, even though it was hard to decide the theme.

For historical purposes find a screenshot of the old theme:
Pintant - old - theme

Bye bye Thinkpad T60

A message for my old IBM Thinkpad T60 laptop: thank you for all your good service! I probably can say that it’s the best laptop that I have ever had and the one that lasted longest (8 years is quite a lot for a laptop!)

In August 2006 I got my previous laptop: an IBM Thinkpad T60. Even though I got it after IBM sold the laptop division to Lenovo it still says IBM Thinkpad. I didn’t know at that time but the laptop became a classic in the laptop world, and many people are building Frankenpads: the case of a T60 or T61 and they change almost all the components (even the screen) to revamp to the current standards. At the moment I don’t plan to have a Frankenpad but who knows in the future (and to be honest, I think that the T60 case is a bit too bulky).

In that 2006 laptop I installed a Debian Sarge that I updated quickly to the Debian Testing that would become Debian Etch. Then I updated it to a Debian Lenny, Squeeze and finally Wheezy (and then since then I kept the system running Debian testing).

Initially I used ReiserFS and then in 2009 I bought a new hard disk (greater capacity) and I moved the system and used XFS and also LUKS to cypher the data.

In 2014 the laptop was becoming quite slow. New webpages required more interaction, photos were bigger, mobiles took more photos, the screen during the last years got dimmer…

So, hello new laptop here!

PS: I haven’t written much here the last weeks… one of the reasons is that I spent time setting up the new laptop, moving the data, etc.

Llibre: Hackstory

Fa poc he llegit el llibre Hackstory de Mercè Molist. Quan el vaig començar no pensava que m’agradés tant com ha estat. Em va sorprendre molt.

És molt amè, informatiu, entretingut. Si recordo bé comença cap a l’any 1980 amb temes de radioaficionats (que vaig fer els meus “pinitus”), després parla de BBS (en vaig ser usuari per força temps), les guerres amb Timofònica, ezines, IRC, estudiants entrant a universitats, Bulma, Badopi…

Alguna gent que surt a Hackstory els he conegut o conec. Molta altra gent els he llegit o em sonaven els noms, o el què havien fet. Recordo fa moltíssims anys llegir algunes revistes com ara SET.

M’ha agradat aprendre més d’aquells temps, de tenir una visió des de l’interior (havia vist el món des de fora però no des de dins). M’ha agradat re-viure algunes coses que havia llegit…

Si per aquí algú li sona el què he escrit més amunt: llegiu el llibre! I hi ha una feinada darrera el llibre que es veu que és immensa!

Tema pràctic: el vaig llegir al meu llibre electrònic en format epub (es pot baixar de (enllaç epub).
A la Web hi ha més informació i es pot llegir també online.

El projecte va ser fundat per crowdfunding. A hores d’ara es poden fer donacions mitjançant Paypal o bé Bitcoins (veure al menú de l’esquerre). Si us ho passeu bé llegint-lo també us ho passareu bé col·laborant una mica.

Així que gràcies Mercé i tothom qui hi ha col·laborat!

Mapnik, osmosis, MongoDB bbox queries

Recently I’ve done a few tests querying data (POIs, Points of Interest) from Open Street Map. I imported the data from OSM into different databases and formats and I wanted to query it.

I’ll show 3 queries: from MongoDB, from PostgreSQL (tables using Osmosis format), from PostgreSQL (tables using osm2psql format).

POIs from MongoDB

I found this one the easiest query to write. I imported the data using the steps described on Derick Rethan’s blogpost (see his Github repository, I used the command:
php5 ./3angle/import-data.php localhost spain-latest.osm to import it.

To query all the POIs for a given bounding box I use:

query = {"l": {"$geoWithin": { "$box": [ [ southwest_lng, southwest_lat], [northeast_lng, northeast_lat] ] } } }
pois = list(pois.find(query))

(replace southwest_lng and the other variables by their positions such as -0.09546518325805664,51.50291256180409,-0.08447885513305664,51.5070930676199 for a small part of London).

This was easy and I liked MongoDB and their GIS (geographical queries) support. It also was my first contact with MongoDB.

POIs from PostgreSQL, Osmosis format

I used osmosis to import data from OSM into PostgreSQL.

The query to get all the POIs for a given bounding box, in my case, is:
SELECT id, tags -> 'amenity' AS amenity, ST_Y(geom) as latitude, ST_X(geom) AS longitude FROM nodes WHERE ST_Intersects(geom, ST_MakeEnvelope(%(southwest_lng)s,%(southwest_lat)s,%(northeast_lng)s,%(northeast_lat)s,4326))

(replace %(southwest_lng)s and the variables by their values, see MongoDB for an example)

Notice that a “\d nodes” (describe nodes) table returns:

tags | hstore |
geom | geometry(Point,4326) |

“tags” column type is hstore (it’s a key-value column). The “geom” type is geometry(Point,4326). 4326 is standard coordinate system for the Earth (see World Geodetic System Wikipedia’s entry).

In this case, the query returns all nodes which geometry (just the position, in this case they are always points) intersects with the given box created using 4326 standard. It’s important that the ST_Intersects projection (4326) matches the projection defined in the column. Forgetting the standard (4326) in the ST_Intersects causes the query to run but not to get the expected results (having some sort of visual validation is important: I thought that the query was working because it returned results… but it wasn’t the case).

POIs from PostgreSQL, osm2psql format

You could import OSM data into PostgreSQL using osm2psql. This format is usually used by Mapnik to render the maps.

In this case the query is:
SELECT osm_id AS id, amenity, ST_Y(ST_Transform(way,4326)) AS latitude, ST_X(ST_Transform(way,4326)) as longitude FROM planet_osm_point WHERE ST_Intersects(way, ST_Transform(ST_MakeEnvelope(%(southwest_lng)s,%(southwest_lat)s,%(northeast_lng)s,%(northeast_lat)s,4326), 900913))

The planet_osm_point_index table contains the position (way column) as:

way | geometry(Point,900913) |

It’s not using the projection 4326 but the projection 900913 (also known as Web Mercator. Wikipedia recognizes that it’s a confusing denomination, and I would like to add that the fact that in PostgreSQL the projections are just numbers instead of names is even more confusing.

In this case the query creates a ST_MakeEnvelope (a bounding box) using the location (using the projection 4326 because my client passes it using this format). After that the function ST_Transform transforms this into the projection 900913 because it’s the one saved in the database (otherwise we might not get the correct results). And, like I did before, the query checks that the POIs are in the bounding box (envelope).

To return the data: it gets the position (way) which converts to the desired projection (ST_Transform) and then returns the latitude or longitude (using ST_X, ST_Y).

I found the PostgreSQL quite confusing because the naming of the functions and a new concept for me was the different projections: if one is not careful they can be easily mixed.

I was surprised that I couldn’t find on the Internet any example queries like these ones: it seems that getting POIs from a bounding box should be the “Hello World!” for OSM and PostgreSQL. So, in this blog post, I hope to fix The Internet adding the missing information.

Foreign travel advice: Spain

A few months ago I was researching how safe it is to travel to a few countries. I used the Foreign Travel Advice webpage from the British government.

I thought to do a bit of “quality control” of their information. What do they say about countries that I know well?

So I went to read what they say for Spain. I could find that in Spain “There is a general threat from terrorism.” (this helps to understand what they mean by “general threat from terrorism” when they say this about other countries). I also learnt that a nurse in Spain tested positive for Ebola (they really update the information often).

But what it surprised me the most is the explicit warning that “balconing” is risky. It says:
“There have been several deaths as a result of falls from balconies. Don’t take any unnecessary risks, especially when under the influence of drink or drugs. See Balcony falls”. It’s so important that it links to a specific section balconing falls which adds more information:

“There have been a number of very serious accidents (some fatal) as a result of falls from balconies. Many of these incidents have been caused by British nationals being under the influence of drink or drugs and most should have been avoidable. Your travel insurance probably won’t cover you for incidents that take place while you’re under the influence of drink or drugs.

Some local councils have introduced laws banning the misuse of balconies with fines for those who are caught.

I’m sure that all the balconing accidents were avoidable!

I’m surprised that this needs to be written there but on the other hand every year a few people die because they jump from a balcony to the swimming pool (usually drunk people). I’m not sure how effective the warning will.

One more public announcement: if you don’t play the lottery and someone sends you an email to say that you won the lottery but that you have to pay some costs: very likely that it’s a scam! (and not only in Spain):

There have been reports of lottery scams in Spain. A person receives what appears to be official notification from the Spanish Inland Revenue office (Hacienda) that they’ve won the Spanish lottery and should deposit money in a bank account to receive their winnings. It’s likely to be a scam if you haven’t entered a lottery, you’re asked to pay anything up-front and the contact telephone number is for a mobile phone.”

Some time ago I did a similar exercise: I read what the Spanish Foreign Travel Advice says about the United Kingdom. Well, I have to say that after reading that text it seems that Oxford Street is one of the most dangerous places in the world!

Llibre: Los Últimos Libres

Fa poc que he acabat el llibre “Los Últimos Libres” de Victor M. Valenzuela. Es pot comprar en format epub o en format paper.

M’ha agradat molt! L’autor és enginyer informàtic, el llibre és fàcil de llegir i m’ha enganxat molt. Es pot llegir en pocs dies ja que és curt i fàcil de llegir.

Si podeu done-li una oportunitat, a veure si us agrada. Va d’una parella que són informàtics i usen programari lliure.

El llibre està situat en un “futur proper” i m’ha agradat com barreja coses que passen avui dia, coses que podrien passar i coses que són més ficció. I tot té prou sentit.

London LOOP finished

Last June I mentioned in another blog post: on the last weekend of April I started the London LOOP (London Outer Orbital Circle).

In the first weekend of October I finished it! 240 Km, 15 stages (officially 23 stages but I followed a book that divided it into 15 stages).

The London LOOP has been my first hike in multiple stages. It visits different parts of Greater London. I started in spring and finished in autumn: I saw the changes in nature (especially the starting/ending point… where I was 5 months later).

Many people have asked about my favourite stage. I’m not sure which is my favourite stage… They are so different! To mention a few random good memories:

  • I walked part of the Grand Union Canal (in Uxbridge). A friend lives there and he joined me on 2 stages
  • I started (and finished!) in Cockfosters: where the Piccadilly Line trains that I use come from or go to
  • I walked through corn fields (yes, in London, just 20 minutes from an underground station there are corn fields)
  • I saw amazing views of London (from South London)
  • I walked in nice woods (I like woods!)
  • A few friends joined me on the way (thanks to all of them!)
  • I walked 20 Km. and finished near a house warming party of a friend… what a good coincidence!
  • I met Derick Rethans: a mapper from Open Street Map that we discovered on IRC that we could meet easily
  • I walked just next to Heathrow airport seeing and hearing the planes (did you know that there is a nice wood just next to Heathrow with a river?)
  • I got really exhausted in a few stages… which was rewarding
  • I went to parts of London that I wouldn’t visit otherwise: I like London and I like to discover unknown parts of London
  • I could see the Queen Elizabeth bridge from far away
  • I crossed Greenwich Meridian twice
  • I closed a circle: arriving to the same point 5 months later

The official stage number 1 (it wasn’t the first stage for me): it’s nice because I arrived

I did the London LOOP in an unusual direction: instead of anti-clockwise I did clockwise. I started at Cockfosters because it's handy to go from my home. And I thought that I would walk to the west: to perhaps visit a friend in Uxbridge after a few stages and then perhaps the Heathrow airport... and I liked it so much that I did it all!

Because I did the unusual direction I could say "hi" to a few London Loopers and it was easier to meet with Derick as I mentioned before.

Sadly the directions from the book were not useful for me (the book is written to walk anti-clockwise) but using the mobile app OruxMaps, the GPS track from Open Street Map and also the sign posts this wasn't a problem.

And guess what? I wanted a break of hiking in loops but I have started the Capital Ring. It’s a similar idea to London LOOP but smaller: closer to the city, shorter stages, but it seems that it goes through interesting bits of London too. So far I like it.

So, if you want: have a look at London LOOP. My favourite resources were Luphen’s webpage and Waymarked Trails.

Python UK Conference 2014: very personal point of view

Like in 2013 I attended the Pycon-UK conference in Coventry. Amazing conference, I went to many talks with different topics: Python itself, high performance Python, testing software, some more social talks, the always interesting lightning talks, etc.

I could write about many of the talks but some people are already doing it in the PostConf Wiki page.

For the second year the conference happens on the Open House London weekend (which is not great, since I have to miss it) and the conference, for second year, is on my birthday.

Last year I thought “oh no, the conference on my birthday?!”. But actually I’m getting used to it, and it’s not too bad.

I spend the birthday with some friends who come to the conference… and many other Python developers, all of them are nice. In a way, I consider the Python-UK my very personal birthday present that I buy for myself.

I also buy some chocolates to enjoy with friends. Next year I might get some more sweets to share more widely after the dinner, this year I didn’t have enough I think.

I would like to say thank you to tall the organizers, sponsors and also the delegates themselves to make this conference quite special! The fact that the schedule is on a Wiki and organized partially on the Wiki shows how open it is.

Anyway, next confirmed conference: FOSDEM!