The UK has a rather odd regime for intercepting on-line communications. The law distinguishes between the content of communications and "communications data" (such as a list of the telephone calls you have made). Content cannot be used in criminal investigations, although it is widely believed that the security services intercept a lot of it as part of their general signals intelligence activity. However "communications data" is available to pretty much any part of Government provided that a sufficiently senior person signs off on the request. No warrant is needed.
The UK Government is consulting on proposals to require ISPs to gather and store a lot more "communications data". At present they basically have to take the IP header data: origin, destination, port number and protocol (TCP/UDP). Volumes and times are likely to be recorded, and of course your DHCP lease so that they can tie a physical machine to the IP address. So with this they can see that you browsed http://www.bbc.co.uk, but not what you actually read. But the Government argues that this is increasingly insufficient. If its investigating suspicious activity, such as a big purchase of fertilizer, then it needs to find out who the suspect is communicating with. If the suspect uses an off-shore email service such as Gmail then the existing communications data regime can't get at this.
So the Government's proposed solution is to require ISPs to capture and store more data, including "third party data", which seems to mean anything inside the packets that helps to identify the people you are communicating with.
The problem with this is that its an impossible job. Most stuff that is even vaguely sensitive (like Gmail) is sent using HTTPS as a matter of course. Even if this could be got over, communication data is simply too slippery a concept once you get away from IP headers. For instance, MMORPGs like Runescape and World of Warcraft (and for that matter Second Life) have "chat" facilities that use game location to determine who can "hear" you. Suppose a conspiracy decides to meet at the edge of a forest in Runescape: their location is exactly the sort of communications data that the Government wants to capture. Doing so requires someone to reverse engineer the existing Runescape protocol and then write software to extract just that part of the data. This software would then have to be run by every ISP in the UK, and the whole exercise repeated for every other multi-player game and interactive website.
This would be a huge effort, but it would still fail to capture data about any criminal who had a modicum of technical knowledge, because there are too many options for setting up channels that the Government cannot intercept. For instance encrypted messages could be left as Usenet posts. Freenet exists specifically for avoiding surveillance. And SSH tunnel services can let you use any Internet service without being tracked.
So in effect the Government is proposing a huge technical boondoggle to do the impossible. Not for the first time, and I'm sure it won't be for the last.
My full response is available here in PDF format.
Saturday, May 2, 2009
Sunday, April 5, 2009
Bruce Sterling accused of sham marriage
I've just read on Bruce Sterling's blog that he and his Serbian wife have been accused of a "green card" marriage by the US immigration authorities. They basically have a couple of weeks to prove to the authorities satisfaction that their marriage is not a sham. If they cannot do so then Jasmina Sterling (nee Tesanovic) will be deported.
This is wrong on many levels. I've never met them, so I can't comment myself on the reality of their marriage, except to note that Bruce Sterling is a good looking wealthy intelligent over-achieving man, so I can't see that he needs to hire himself out as a husband-of-convenience, nor does it seem likely that he needed to offer a green card just to get a woman into his bed.
(Aside: I also think it truly ironic that an organisation that takes 6 months to decide on a 3-month visitor's visa allows only a couple of weeks for them to put together evidence that their marriage is real).
But US Immigration don't think like that. It seems from what Sterling wrote that they check up on marriages between Americans and non-citizens, and look for the paper trail that would be generated by people living together: joint ownership of homes, bank accounts and the like. And because of their somewhat nomadic existence it seems that Bruce and Jasmina don't have that paper trail.
Students of organisational behaviour will recognise an anti-pattern here, although I don't know if its ever been written down. It looks like this:
Name: Chasing Reality
Context: Enforcement of rules that mandate an ill-defined threshold of performance.
Forces: People attempt to meet the performance threshold with minimum effort. The enforcing authority becomes concerned that their effectiveness is being eroded.
Supposed solution: The enforcing authority requires increasingly stringent evidence of performance. The rules about what is "sufficient" may also be kept secret or poorly defined to prevent people from engineering their evidence to follow the requirements.
Resulting context: the enforcing authority becomes increasingly unable to judge real performance, and instead becomes obsessed with the production of evidence. Success depends more on finding out what the real criteria are and meeting them than on actually performing the original task.
I have seen this pattern before, although never with such horrendous consequences. Some years ago a senior manager told me that he had a set of criteria for any application to spend money, but he would not tell anyone what they were for fear that all the applications would then meet those criteria. When I visited the US many years ago, before the visa waiver programme, I had to provide evidence of solvency to get a visa, but was not told what "evidence of solvency" looked like (I photocopied a year of bank statements; it did the trick).
Mr & Mrs Sterling are apparently required to meet a secret (or at least highly under-advertised) set of criteria for a "real" marriage. When the US first controlled immigration I expect it only had to be a legal marriage, but sham marriages were an obvious problem. So the Immigration people wanted to see wedding photos. So sham couples hired photographers. Then Immigration wanted to see you living together, so sham couples rented apartments. And so it goes on. The Immigration people are on an endless treadmill: whatever evidence they require will be produced, whether by sham couples or real ones. Now its got to the point where the sham couples will actually have better evidence of their reality than real couples, because they are more aware of the need to generate it. Or as Granny Weatherwax once put it, "Things that try to look like things often look more like things than things."
Its also ironic that a writer with a great talent for spotting trends and capturing the zeitgeist should be caught up in exactly the kind of insane socio-political trend that he writes about so entertainingly.
This is wrong on many levels. I've never met them, so I can't comment myself on the reality of their marriage, except to note that Bruce Sterling is a good looking wealthy intelligent over-achieving man, so I can't see that he needs to hire himself out as a husband-of-convenience, nor does it seem likely that he needed to offer a green card just to get a woman into his bed.
(Aside: I also think it truly ironic that an organisation that takes 6 months to decide on a 3-month visitor's visa allows only a couple of weeks for them to put together evidence that their marriage is real).
But US Immigration don't think like that. It seems from what Sterling wrote that they check up on marriages between Americans and non-citizens, and look for the paper trail that would be generated by people living together: joint ownership of homes, bank accounts and the like. And because of their somewhat nomadic existence it seems that Bruce and Jasmina don't have that paper trail.
Students of organisational behaviour will recognise an anti-pattern here, although I don't know if its ever been written down. It looks like this:
Name: Chasing Reality
Context: Enforcement of rules that mandate an ill-defined threshold of performance.
Forces: People attempt to meet the performance threshold with minimum effort. The enforcing authority becomes concerned that their effectiveness is being eroded.
Supposed solution: The enforcing authority requires increasingly stringent evidence of performance. The rules about what is "sufficient" may also be kept secret or poorly defined to prevent people from engineering their evidence to follow the requirements.
Resulting context: the enforcing authority becomes increasingly unable to judge real performance, and instead becomes obsessed with the production of evidence. Success depends more on finding out what the real criteria are and meeting them than on actually performing the original task.
I have seen this pattern before, although never with such horrendous consequences. Some years ago a senior manager told me that he had a set of criteria for any application to spend money, but he would not tell anyone what they were for fear that all the applications would then meet those criteria. When I visited the US many years ago, before the visa waiver programme, I had to provide evidence of solvency to get a visa, but was not told what "evidence of solvency" looked like (I photocopied a year of bank statements; it did the trick).
Mr & Mrs Sterling are apparently required to meet a secret (or at least highly under-advertised) set of criteria for a "real" marriage. When the US first controlled immigration I expect it only had to be a legal marriage, but sham marriages were an obvious problem. So the Immigration people wanted to see wedding photos. So sham couples hired photographers. Then Immigration wanted to see you living together, so sham couples rented apartments. And so it goes on. The Immigration people are on an endless treadmill: whatever evidence they require will be produced, whether by sham couples or real ones. Now its got to the point where the sham couples will actually have better evidence of their reality than real couples, because they are more aware of the need to generate it. Or as Granny Weatherwax once put it, "Things that try to look like things often look more like things than things."
Its also ironic that a writer with a great talent for spotting trends and capturing the zeitgeist should be caught up in exactly the kind of insane socio-political trend that he writes about so entertainingly.
Saturday, January 31, 2009
Bad banking drives out good
I heard this quote from the CEO of Standard Chartered Bank on Radio 4's Today programme this morning, and it summarises the problem very well. (Its also a reference to Gresham's Law.)
Its become a commonplace to simply blame the credit crunch on "stupid, greedy bankers". The trouble is, suppose you were a smart, frugal banker five years ago. Your exceptional brains let you see that subprime self-certified mortgages were a bad idea, and therefore anything built on them was clearly going to collapse. Therefore you weren't going to invest in that part of the market.
The trouble is, you would have had years of below-average results. Your clients and employers would be looking at your competitors and pointing out how bad your results were. Even if your clients were prepared to listen to your explanations, many are under a legal obligation to maximise their investment returns, and their customers will be less sophisticated and correspondingly less willing to listen to complicated explanations about why their pension is so much less than their neighbours.
So you get removed from your job, and someone else is put in who is willing to do what it takes to deliver market returns.
Looked at from an abstract point of view the game resembles a "Tragedy of the Commons". A banker who gets good returns while taking hidden risks is like the commoner who puts an extra cow on the common pasture. He gets richer while all his neigbours get a little poorer. So everyone else is forced to do likewise, even though everyone can see that collectively this is going to lead to ruin. And so the game goes.
The traditional solutions to the Tragedy of the Commons are:
So without this, what can we do to prevent the next big bust? It doesn't look like there is anything. As memories of this bust fade and the people who experienced it retire, a new generation of bankers will come along with a new ingenious method for hiding risk, and once again bad banking will drive out the good until the whole thing collapses once more.
Its become a commonplace to simply blame the credit crunch on "stupid, greedy bankers". The trouble is, suppose you were a smart, frugal banker five years ago. Your exceptional brains let you see that subprime self-certified mortgages were a bad idea, and therefore anything built on them was clearly going to collapse. Therefore you weren't going to invest in that part of the market.
The trouble is, you would have had years of below-average results. Your clients and employers would be looking at your competitors and pointing out how bad your results were. Even if your clients were prepared to listen to your explanations, many are under a legal obligation to maximise their investment returns, and their customers will be less sophisticated and correspondingly less willing to listen to complicated explanations about why their pension is so much less than their neighbours.
So you get removed from your job, and someone else is put in who is willing to do what it takes to deliver market returns.
Looked at from an abstract point of view the game resembles a "Tragedy of the Commons". A banker who gets good returns while taking hidden risks is like the commoner who puts an extra cow on the common pasture. He gets richer while all his neigbours get a little poorer. So everyone else is forced to do likewise, even though everyone can see that collectively this is going to lead to ruin. And so the game goes.
The traditional solutions to the Tragedy of the Commons are:
- Property rights. Divide up the commons into individual plots and allocate them to the commoners. Failing that, give each commoner a tradeable right to put exactly one cow on the commons.
- Regulation. Make a law that each person can only put one cow on the commons.
So without this, what can we do to prevent the next big bust? It doesn't look like there is anything. As memories of this bust fade and the people who experienced it retire, a new generation of bankers will come along with a new ingenious method for hiding risk, and once again bad banking will drive out the good until the whole thing collapses once more.
Monday, January 26, 2009
Wikipedia and Britannica are Converging
Is there a network equivalent of market forces? The crowdsource equivalent (or perhaps generalisation) of Adam Smith's "Invisible Hand"?
When I was in my early teens my school got the new edition of the Encyclopaedia Britannica in over 30 volumes costing around £1,000. It was a huge wealth of authoritative information on just about everything. I spent quite a few lunch hours just reading about stuff.
In the 90s Microsoft bought out Encarta on CD-ROM. It was a lot less information, but it cost a lot less than £1,000. Britannica has been in trouble ever since. Now there is Wikipedia, which is free, a bit less authoritative than Britannica, but has even more information and a great deal more mindshare.
So Britannica has responded by taking a page out of Jimbo Wales's book; its going to start accepting user-generated content, albeit with a stiff editorial barrier to acceptance into the core Encyclopaedia.
Meanwhile the latest controversy on Wikipedia is about whether user-generated content needs more controls. I say "more" because Wikipedia has always had some limits; locked pages, banned editors, and in extreme cases even the deletion of changes (as opposed to reversion, which preserves them in the page history). Increasingly sophisticated editorial policies have been put in place, such as "No Original Research" (originally put in to stop crank science, but now an important guiding principle).
When you look at Wikipedia's history there is a clear trend; every so often Wikipedia reaches a threshold of size and importance where its current level of openness doesn't work. At this point Jimbo Wales adds a minimal level of editorial control. Locked pages, for instance, were added because certain pages attracted unmanageably high levels of vandalism.
It seems that Wikipedia and Britannica are actually converging on the same model from different ends of the spectrum: Wikipedia started out completely open and is having to impose controls to manage quality. Meanwhile the Britannica started out completely closed and is having to allow in user-generated content because comissioning experts to write it is too expensive. Somewhere in the middle is a happy medium that gets good quality content without having to charge subscribers.
Not charging is important. In these days of the hyperlink, an encyclopedia is not just a source of knowledge, it is a medium of communication. If I want to tell people that they should read about the history of the Britannica then I refer to the Wikipedia page because very few people who read this are going to be able to follow a link to a Britannica article (even if I were a subscriber, which I am not).
Wikipedia is often said to have been inspired by the "open source" model in that anyone can edit Wikipedia just as anyone can edit the Linux source code. In fact the cases are not parallel. The GPL allows me to download a copy of the Linux source code, hack it to my heart's content, and give copies of my new version to anyone who wants them. What it does not do is authorise me to upload my version to kernel.org and pass it off as the work of Linus Torvalds. Getting my changes into the official kernel means passing through a strict quality assurance process including peer review and extensive testing on multiple architectures.
So I think that this proposal to create "flagged revisions" for editorial review moves Wikipedia towards the open source model rather than away from it. Anyone will always be able to fork Wikipedia if they wish: the license guarantees it. But the offical version at wikipedia.org will earn increasing trust as the quality assurance improves, just as the official version of the Linux kernel is trusted because of its quality assurance.
When I was in my early teens my school got the new edition of the Encyclopaedia Britannica in over 30 volumes costing around £1,000. It was a huge wealth of authoritative information on just about everything. I spent quite a few lunch hours just reading about stuff.
In the 90s Microsoft bought out Encarta on CD-ROM. It was a lot less information, but it cost a lot less than £1,000. Britannica has been in trouble ever since. Now there is Wikipedia, which is free, a bit less authoritative than Britannica, but has even more information and a great deal more mindshare.
So Britannica has responded by taking a page out of Jimbo Wales's book; its going to start accepting user-generated content, albeit with a stiff editorial barrier to acceptance into the core Encyclopaedia.
Meanwhile the latest controversy on Wikipedia is about whether user-generated content needs more controls. I say "more" because Wikipedia has always had some limits; locked pages, banned editors, and in extreme cases even the deletion of changes (as opposed to reversion, which preserves them in the page history). Increasingly sophisticated editorial policies have been put in place, such as "No Original Research" (originally put in to stop crank science, but now an important guiding principle).
When you look at Wikipedia's history there is a clear trend; every so often Wikipedia reaches a threshold of size and importance where its current level of openness doesn't work. At this point Jimbo Wales adds a minimal level of editorial control. Locked pages, for instance, were added because certain pages attracted unmanageably high levels of vandalism.
It seems that Wikipedia and Britannica are actually converging on the same model from different ends of the spectrum: Wikipedia started out completely open and is having to impose controls to manage quality. Meanwhile the Britannica started out completely closed and is having to allow in user-generated content because comissioning experts to write it is too expensive. Somewhere in the middle is a happy medium that gets good quality content without having to charge subscribers.
Not charging is important. In these days of the hyperlink, an encyclopedia is not just a source of knowledge, it is a medium of communication. If I want to tell people that they should read about the history of the Britannica then I refer to the Wikipedia page because very few people who read this are going to be able to follow a link to a Britannica article (even if I were a subscriber, which I am not).
Wikipedia is often said to have been inspired by the "open source" model in that anyone can edit Wikipedia just as anyone can edit the Linux source code. In fact the cases are not parallel. The GPL allows me to download a copy of the Linux source code, hack it to my heart's content, and give copies of my new version to anyone who wants them. What it does not do is authorise me to upload my version to kernel.org and pass it off as the work of Linus Torvalds. Getting my changes into the official kernel means passing through a strict quality assurance process including peer review and extensive testing on multiple architectures.
So I think that this proposal to create "flagged revisions" for editorial review moves Wikipedia towards the open source model rather than away from it. Anyone will always be able to fork Wikipedia if they wish: the license guarantees it. But the offical version at wikipedia.org will earn increasing trust as the quality assurance improves, just as the official version of the Linux kernel is trusted because of its quality assurance.
Wednesday, January 21, 2009
The next challenge for Linux
I was in the local branch of "Currys" (UK electrical and electronic goods chain) recently and had a look at the line-up of computers. They had some little netbooks, and taped next to each one was a little note saying something to the effect of "This runs Linux, so it won't run Windows software". It was a local version of a wider story about Linux:
Of course those of us who use Linux know different. But for most people, Linux is something they might have heard about once or twice, but they didn't pay any attention and couldn't tell you anything about it. Outside its current user base and their immediate circle of friends and family, Linux has zero mindshare.
This is not another lament about Joe Sixpack being too stupid to understand Linux. The problem is not that Linux is too complicated, its that Linux and Windows do things differently. Imagine someone who was raised on Linux; how would they react to Windows? Software installation would seem complicated and geeky, the single desktop would feel claustrophobic, and as for the idea of paying for software...
So I think we need to sort out a message to broadcast to the world and then focus on it. I suggest the following:
- People buy netbooks and then discover that Windows isn't part of the bundle, and they don't like it.
- A teacher found a student handing out Linux CDs and accused him of pirating software.
- A student accidentally bought a Dell laptop with Ubuntu instead of Windows, and complained that she couldn't submit the Word documents required by her college (they say they are happy to accept OpenOffice.org documents) and the installation CD for her broadband service wouldn't install so she couldn't get the Internet (Ubuntu connects to the Internet just fine without it). The local TV station picked up the story as a "consumer rights" case and was amazed to find a virtual lynch mob chasing them for being less than enthusiastic about Ubuntu. They quoted an expert saying that Ubuntu "isn't for everyone" and is more suited to tinkerers than people who just want to get stuff done.
- There are two sorts of computers: Apple Macs, and PCs.
- Apple Macs run Apple software. PCs run Windows software.
- Windows is bundled with PCs. On expensive PCs the bundle might include Office as well.
- If you want extra software, you buy it at a store, take it home and stick the CD in the drive.
Of course those of us who use Linux know different. But for most people, Linux is something they might have heard about once or twice, but they didn't pay any attention and couldn't tell you anything about it. Outside its current user base and their immediate circle of friends and family, Linux has zero mindshare.
This is not another lament about Joe Sixpack being too stupid to understand Linux. The problem is not that Linux is too complicated, its that Linux and Windows do things differently. Imagine someone who was raised on Linux; how would they react to Windows? Software installation would seem complicated and geeky, the single desktop would feel claustrophobic, and as for the idea of paying for software...
So I think we need to sort out a message to broadcast to the world and then focus on it. I suggest the following:
- Linux is an alternative to Windows and Apple. It comes in several flavours.
- Linux isn't easier or harder than Windows, but it is different, and it takes a while to get used to those differences.
- Linux is bundled with a huge range of downloadable free software, including office, graphics and photographic packages.
Sunday, December 14, 2008
Where did all the money go?
I've got a simple question about the Credit Crunch? Where did all the money go?
I've always thought of money as something that is "conserved" in the physics sense, like energy. Money in = money out + money stored. So when someone "loses" money it implies that someone else got it. If I run a company that loses money, what it means is that I'm getting less money in than I'm spending on staff and supplies. But the money hasn't disappeared, its just moved from my pockets to my suppliers (and their suppliers, and so on).
But the financial institutions seem to have had billions of dollars disappear into thin air. They got poorer, sometimes so poor that they went bust, but nobody seems to have become correspondingly richer.
I guess this has something to do with fractional reserve banking, which I know doesn't conserve money. Suppose I start with £1000 and put it in a bank. The bank shows a balance of £1000, but it doesn't just hang on to my money, it loans £500 of it to Joe Bloggs, who spends it on a new TV, and the person who sold the TV also puts the money in the bank. So now the bank has $1,500 on deposit even though our imaginary economy just started with £1,000. So what happens if Joe can't pay the money back?
Can someone enlighten me?
I've always thought of money as something that is "conserved" in the physics sense, like energy. Money in = money out + money stored. So when someone "loses" money it implies that someone else got it. If I run a company that loses money, what it means is that I'm getting less money in than I'm spending on staff and supplies. But the money hasn't disappeared, its just moved from my pockets to my suppliers (and their suppliers, and so on).
But the financial institutions seem to have had billions of dollars disappear into thin air. They got poorer, sometimes so poor that they went bust, but nobody seems to have become correspondingly richer.
I guess this has something to do with fractional reserve banking, which I know doesn't conserve money. Suppose I start with £1000 and put it in a bank. The bank shows a balance of £1000, but it doesn't just hang on to my money, it loans £500 of it to Joe Bloggs, who spends it on a new TV, and the person who sold the TV also puts the money in the bank. So now the bank has $1,500 on deposit even though our imaginary economy just started with £1,000. So what happens if Joe can't pay the money back?
Can someone enlighten me?
Wednesday, November 5, 2008
Fundamentalist terrorist assassination plot defeated
Tonight the British people celebrate the defeat of a terrorist assassination plot. A group of disaffected men, followers of a foreign religion and egged on by propaganda from overseas and a radical cleric, planned a spectacular atrocity: they would blow up the entire government in one huge explosion. The plot was foiled by the security services, who, responding to a tip-off, mounted a surveillance operation and arrested one of the plotters red handed as he prepared to detonate the explosives. The rest of the gang were tracked down and shot or arrested when soldiers stormed their hideout. In accordance with an executive order from the head of state the survivors were subsequently imprisoned, tortured, and executed after a show-trial.
This happened over 400 years ago, on November 5th 1605. The plotters were Catholics rather than Muslims, but apart from that the story sounds disconcertingly modern, although we tend not to go in for show trials these days: they went out of fashion with Stalin.
Today the Gunpowder Plot is celebrated as Bonfire Night: Guy Fawkes (the one caught with the explosives) is still burned in effigy, and in some places the Pope gets the same treatment as well. However, over the past 400 years the anti-Catholic content has almost entirely leached away: now its just an excuse to let off some fireworks. A Catholic can join in the celebrations just as I (a Briton) joined in the 4th of July celebrations when I was living in the US.
Maybe one day we will look back on 9/11 the same way. I certainly hope so.
This happened over 400 years ago, on November 5th 1605. The plotters were Catholics rather than Muslims, but apart from that the story sounds disconcertingly modern, although we tend not to go in for show trials these days: they went out of fashion with Stalin.
Today the Gunpowder Plot is celebrated as Bonfire Night: Guy Fawkes (the one caught with the explosives) is still burned in effigy, and in some places the Pope gets the same treatment as well. However, over the past 400 years the anti-Catholic content has almost entirely leached away: now its just an excuse to let off some fireworks. A Catholic can join in the celebrations just as I (a Briton) joined in the 4th of July celebrations when I was living in the US.
Maybe one day we will look back on 9/11 the same way. I certainly hope so.
Subscribe to:
Posts (Atom)