Posts Tagged ‘nutch’

Mashups and MashupCamp Dublin

Wednesday, November 14th, 2007

MashupCamp was held in Dublin last weekend but unfortunately I didn’t get a chance to attend. Trying to fit it in around having a personal life, with the day job during the week and hacking on the website is impossible. I suppose I am more an observer from a far in relation to these type of events. I would love to get along to some of the other similar events such as Barcamp Ireland and OpenCoffee Dublin I am a real fan of the culture around these events and the type of energy they generate hopefully I get to one in the future.

By all accounts MashupCamp went pretty well with a large international attendance with some of us Irish turning up as well (maybe not in the numbers we should have!). I find the whole area of mashups very interesting it is and will be one of the key areas for the next wave of webapps.

For me the main thing required from websites to allow mashups to be build round them is access to data. I think that Tim O’Reilly hit the nail on the head with his expression “its the data stupid!“. With greater data mobilitly we will be able to build some really exiting web apps in the future. Just look at the great mashups alright build on existing available APIs now imagine how these will evolve as more data becomes available.

How does this relate to pricefindr? well I hope in the future to add the same data accessibiltiy and mobility to pricefindr although not much demand for it now (traffic isn’t exactly setting the world on fire!) who knows in the future maybe we’ll have a whole set of mashups based around pricefindr data. To be honest I don’t think it would take much of an effort to start offering some of these types of services thanks to nutch and its support for OpenSearch RSS. Something else to look into when i get some time…

Technologies used…

Thursday, November 1st, 2007

Just to give a flavor of some of the technologies that go into building pricefindr. For the day job as an engineer most of my development is in Java/J2EE building enterprise systems. I have been using java for over 10 years so you won’t be to surprised to know that the majority of pricefindr is developed using java. A number of excellent open source projects go into the site, the main ones being

  • lucene (used as the search engine, it is a full-featured text search engine library written entirely in Java)
  • nutch (build on top of lucene, used to crawl and index the various websites)
  • jetty (an excellent lightweight HTTP web server and Servlet Container, the website is powered by it)
  • springframework (used as the glue, pulls together the various components behind the scenes)
  • ubuntu (the operating system behind it all, a great Linux distribution)

There are many other bits and pieces behind the scenes but don’t want to give a geek overload right now. I’ll expand and go into more detail on some of the above in the future.