Tom Morris

To illustrate some of the things I spoke about at BarCamp, I have been putting together a really simple parser for (X)HTML pages contaiing microformats to turn them in to RDF.

It basically takes a page, runs it through Tidy, runs all the GRDDL stylesheets across it, loads the result in to RAP and outputs the result as (somewhat messy) RDF/XML.

Currently, the parser is reading hCard, hCalendar, XFN, DC-Extract (not a microformat with a capital 'M', but still parseable) and rel-licence. I also have support planned for hReview, GeoURLs and hDOAP.

What's neat about the parser? Once the stuff is in a document, you can easily run SPARQL queries over the pages. One of the planned functions that I hope to add is an RSS adder. What this would do is let you request that all the URLs that are likely to be web pages be checked to see if there is an RSS feed attached which we might be able to add.

What is nice about RDF is that it becomes almost like a universal format, and it is trivially easy to get data out of it.

I'm planning to add some new functions to it in order to extract more data from different places - Flickr parsing, better Twitter parsing. Now I basically have a workflow which I can edit based in PHP5. For instance, Twitter supports XFN, but there is better data available by writing a domain-specific parser for it. It can be as simple as writing:

if (strstr($url, "twitter.com") { $stylesheets[] = // twitter parser

Of course, we can use weak string matching for speeding up the process:

if (strstr($data, "vcard") { // add hCard parser

This is simply to reduce resource usage - parsing stylesheets isn’t the quickest of processes.

The parser itself is 37 lines of PHP5. It’ll grow as I add domain-specific and site-specific conditionals.

You can access the microformat-to-RDF parser at:

xml.opiumfield.com/mf/rdf/URL

The mf/rdf means that it may be possible to start offering other parsing possibilities - mf/xml, mf/rss etc.

If there’s a problem, either post a comment or come and chat - I’ll be in #swig and #microformats most of this evening.

This brings me on to another little service I’ve started offering over REST which is Tidy.

Tidy is a fantastically powerful C application that takes badly marked up (X)HTML documents and tries to make them slightly more sensible - with validation, XML well-formedness and the such being the end result.

If you use the microformat to RDF parser, I run the HTML I get through Tidy anyway, so there’s no need to bother doing it for that.

I am using Tidy because I really like using XSLT, and XSLT doesn’t run on HTML - of course, Beautiful Soup can be used when XSLT doesn’t.

The REST interface for Tidy is:

tools.opiumfield.com/tidy/URL

By default, my Tidy interface returns XHTML (even if not provided). This is an utterly greedy mostly practical, somewhat philosophical decision - I need XML. Non-XML standards don’t deserve existence unless absolutely necessary.

But if you don’t want automatic XHTML conversion to take place, use:

tools.opiumfield.com/tidy/h/URL

tools.opiumfield.com/tidy/html/URL

Yes, both ‘h’ and ‘html’ work fine.

Try not to bombard my server, and try to cache results wherever practical. And if you are in the position to provide similar services, then please do so - drop me an e-mail and I’ll point some of my external traffic to your script.

Tags: rdf, semweb, microformats, grddl, barcamplondon2, tidy, xhtml, html, xml

Comments | TrackBack

Tony Blair - or someone writing on his behalf - has responded to the “scrap the proposed introduction of ID cards” petition which appeared in my inbox. Here it is in full, with some comments (if you don’t see it in your RSS reader, just click through):

The petition calling for the Government to abandon plans for a National ID Scheme attracted almost 28,000 signatures - one of the largest responses since this e-petition service was set up. So I thought I would reply personally to those who signed up, to explain why the Government believes National ID cards, and the National Identity Register needed to make them effective, will help make Britain a safer place.

The petition disputes the idea that ID cards will help reduce crime or terrorism. While I certainly accept that ID cards will not prevent all terrorist outrages or crime, I believe they will make an important contribution to making our borders more secure, countering fraud, and tackling international crime and terrorism. More importantly, this is also what our security services - who have the task of protecting this country - believe.

So I would like to explain why I think it would be foolish to ignore the opportunity to use biometrics such as fingerprints to secure our identities. I would also like to discuss some of the claims about costs - particularly the way the cost of an ID card is often inflated by including in estimates the cost of a biometric passport which, it seems certain, all those who want to travel abroad will soon need.

In contrast to these exaggerated figures, the real benefits for our country and its citizens from ID cards and the National Identity Register, which will contain less information on individuals than the data collected by the average store card, should be delivered for a cost of around £3 a year over its ten-year life.

But first, it’s important to set out why we need to do more to secure our identities and how I believe ID cards will help. We live in a world in which people, money and information are more mobile than ever before. Terrorists and international criminal gangs increasingly exploit this to move undetected across borders and to disappear within countries. Terrorists routinely use multiple identities - up to 50 at a time. Indeed this is an essential part of the way they operate and is specifically taught at Al-Qaeda training camps. One in four criminals also uses a false identity. ID cards which contain biometric recognition details and which are linked to a National Identity Register will make this much more difficult.

Secure identities will also help us counter the fast-growing problem of identity fraud. This already costs £1.7 billion annually. There is no doubt that building yourself a new and false identity is all too easy at the moment. Forging an ID card and matching biometric record will be much harder.

I also believe that the National Identity Register will help police bring those guilty of serious crimes to justice. They will be able, for example, to compare the fingerprints found at the scene of some 900,000 unsolved crimes against the information held on the register. Another benefit from biometric technology will be to improve the flow of information between countries on the identity of offenders.

The National Identity Register will also help improve protection for the vulnerable, enabling more effective and quicker checks on those seeking to work, for example, with children. It should make it much more difficult, as has happened tragically in the past, for people to slip through the net.

Proper identity management and ID cards also have an important role to play in preventing illegal immigration and illegal working. The effectiveness on the new biometric technology is, in fact, already being seen. In trials using this technology on visa applications at just nine overseas posts, our officials have already uncovered 1,400 people trying illegally to get back into the UK.

Nor is Britain alone in believing that biometrics offer a massive opportunity to secure our identities. Firms across the world are already using fingerprint or iris recognition for their staff. France, Italy and Spain are among other European countries already planning to add biometrics to their ID cards. Over 50 countries across the world are developing biometric passports, and all EU countries are proposing to include fingerprint biometrics on their passports. The introduction in 2006 of British e-passports incorporating facial image biometrics has meant that British passport holders can continue to visit the United States without a visa. What the National Identity Scheme does is take this opportunity to ensure we maximise the benefits to the UK.

These then are the ways I believe ID cards can help cut crime and terrorism. I recognise that these arguments will not convince those who oppose a National Identity Scheme on civil liberty grounds. They will, I hope, be reassured by the strict safeguards now in place on the data held on the register and the right for each individual to check it. But I hope it might make those who believe ID cards will be ineffective reconsider their opposition.

If national ID cards do help us counter crime and terrorism, it is, of course, the law-abiding majority who will benefit and whose own liberties will be protected. This helps explain why, according to the recent authoritative Social Attitudes survey, the majority of people favour compulsory ID cards.

I am also convinced that there will also be other positive benefits. A national ID card system, for example, will prevent the need, as now, to take a whole range of documents to establish our identity. Over time, they will also help improve access to services.

The petition also talks about cost. It is true that individuals will have to pay a fee to meet the cost of their ID card in the same way, for example, as they now do for their passports. But I simply don’t recognise most claims of the cost of ID cards. In many cases, these estimates deliberately exaggerate the cost of ID cards by adding in the cost of biometric passports. This is both unfair and inaccurate.

As I have said, it is clear that if we want to travel abroad, we will soon have no choice but to have a biometric passport. We estimate that the cost of biometric passports will account for 70% of the cost of the combined passports/id cards. The additional cost of the ID cards is expected to be less than £30 or £3 a year for their 10-year lifespan. Our aim is to ensure we also make the most of the benefits these biometric advances bring within our borders and in our everyday lives.

Yours sincerely,

Tony Blair

There are a lot of other issues, but this doesn’t really cover the cost correctly. ID cards will cost us a lot more than the £30 - they’ll cost us in implementation costs, which will come out of taxation. They’ll cost us indirectly in adoption costs.

Tags: tonyblair, id cards, no2id

Comments | TrackBack