tag:blogger.com,1999:blog-59755240068248628042024-03-09T02:16:28.148-08:00Paul's PontificationsRandom musings on politics, economics and softwarePaul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.comBlogger69125tag:blogger.com,1999:blog-5975524006824862804.post-65659240999137858642020-03-30T00:32:00.001-07:002020-03-30T00:34:53.401-07:00The Diametric Safety Case ManagerIn my work life I have spent the past two years writing the <a href="https://diametricsoftware.com/">Diametric Safety Case Manager</a> (DSM). This uses <a href="https://www.goalstructuringnotation.info/">Goal Structuring Notation</a> to represent <a href="https://diametricsoftware.com/diagrams">safety arguments</a> and bow-tie diagrams to represent the <a href="https://diametricsoftware.com/dsm-as-hazard-log">relationships between events and hazards</a>.<br />
<br />
Underlying these diagrams is a <a href="https://diametricsoftware.com/dsm-as-hazard-log">safety model</a>; everything on a diagram is represented by an entity in the underlying model. A graphical query notation lets the user create tables and reports based on the contents of the model. Here is an example showing how a <a href="https://diametricsoftware.com/building-a-fmea">Failure Modes and Effects Analysis (FMEA)</a> can be constructed from data entered in bow-tie diagrams.<br />
<br />
The DSM is written in <a href="https://www.haskell.org/">Haskell</a> using GTK3 and Reactive Banana. I've blogged about the underlying mechanism <a href="https://paulspontifications.blogspot.com/2018/02/haskell-with-reactive-banana-and-gtk3.html">here</a>.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com0tag:blogger.com,1999:blog-5975524006824862804.post-34198697860575960092018-07-13T15:21:00.000-07:002018-07-13T15:21:05.643-07:00Donald Trump is Zaphod Beeblebrox<div class="tr_bq">
Here are some quotes from the Hitchhikers Guide to the Galaxy, lightly edited...</div>
<br />
<blockquote class="tr_bq">
One of the major difficulties <strike>Trillian</strike> Theresa May experienced in her relationship
with <strike>Zaphod</strike> Donald Trump was learning to distinguish between him pretending to be
stupid just to get people off their guard, pretending to be stupid
because he couldn’t be bothered to think and wanted someone else to do
it for him, pretending to be outrageously stupid to hide the fact that
he actually didn’t understand what was going on, and really being
genuinely stupid. </blockquote>
<br />
<blockquote>
It is a well known fact that those people who most want to rule people are, ipso facto, those least suited to do it. To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job.</blockquote>
<br />
<blockquote class="tr_bq">
The President in particular is very much a figurehead — he
wields no real power whatsoever. He is apparently chosen by the
<strike>government</strike> voters, but the qualities he is required to display are not those of
leadership but those of finely judged outrage. For this reason the
President is always a controversial choice, always an infuriating but
fascinating character. His job is not to wield power but to draw
attention away from it. On those criteria <strike>Zaphod Beeblebrox</strike> Donald Trump is one of
the most successful Presidents the <strike>Galaxy</strike> United States has ever had — <strike>he has already spent two of his ten presidential years in prison for fraud..</strike></blockquote>
Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com1tag:blogger.com,1999:blog-5975524006824862804.post-76973828186512774482018-04-18T00:23:00.002-07:002018-04-18T00:23:28.490-07:00A very long Haskell typeFor GUI programming I've migrated from the <a href="https://hackage.haskell.org/package/gtk3">gtk2hs</a> library to the <a href="https://hackage.haskell.org/package/gtk3">gi-gtk</a> library. This uses <a href="https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/OverloadedLabels">overloaded labels</a> to access the properties of GTK3 objects.<br />
<br />
I wanted a function which would set the background and foreground colours of a widget. Rather than simply have it modify the object I decided to have it return a list of property setters. It looks like this (the function "contrastText" just returns either white or black):<br />
<span style="font-size: x-small;"><br /></span>
<span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;">setColour c = do<br /> bg <- colourToRGBA c<br /> fg <- colourToRGBA $ contrastText c<br /> return [#backgroundRgba := bg, #foregroundRgba := fg]</span></span><br />
<br />
However when I couldn't get it to type-check I asked ghci what the type should be. Here is the reply!<br />
<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">> :type setColour<br />setColour<br /> :: (Data.GI.Base.Attributes.AttrOpIsAllowed<br /> tag<br /> (Data.GI.Base.Attributes.AttrAllowedOps<br /> (Data.GI.Base.Overloading.FindElement<br /> "foregroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "foregroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")))<br /> (Data.GI.Base.Attributes.AttrLabel<br /> (Data.GI.Base.Overloading.FindElement<br /> "foregroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "foregroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")))<br /> (Data.GI.Base.Attributes.AttrOrigin<br /> (Data.GI.Base.Overloading.FindElement<br /> "foregroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "foregroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")))<br /> obj<br /> ~<br /> 'Data.GI.Base.Attributes.OpIsAllowed,<br /> Data.GI.Base.Attributes.AttrOpIsAllowed<br /> tag<br /> (Data.GI.Base.Attributes.AttrAllowedOps<br /> (Data.GI.Base.Overloading.FindElement<br /> "backgroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "backgroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")))<br /> (Data.GI.Base.Attributes.AttrLabel<br /> (Data.GI.Base.Overloading.FindElement<br /> "backgroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "backgroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")))<br /> (Data.GI.Base.Attributes.AttrOrigin<br /> (Data.GI.Base.Overloading.FindElement<br /> "backgroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "backgroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")))<br /> obj<br /> ~<br /> 'Data.GI.Base.Attributes.OpIsAllowed,<br /> Data.GI.Base.Attributes.AttrInfo<br /> (Data.GI.Base.Overloading.FindElement<br /> "foregroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "foregroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")),<br /> Data.GI.Base.Attributes.AttrInfo<br /> (Data.GI.Base.Overloading.FindElement<br /> "backgroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "backgroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217.")),<br /> Control.Monad.IO.Class.MonadIO m,<br /> Data.GI.Base.Overloading.HasAttributeList obj,<br /> Data.GI.Base.Attributes.AttrBaseTypeConstraint<br /> (Data.GI.Base.Overloading.FindElement<br /> "backgroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "backgroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217."))<br /> obj,<br /> Data.GI.Base.Attributes.AttrSetTypeConstraint<br /> (Data.GI.Base.Overloading.FindElement<br /> "backgroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "backgroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217."))<br /> GI.Gdk.Structs.RGBA.RGBA,<br /> Data.GI.Base.Attributes.AttrBaseTypeConstraint<br /> (Data.GI.Base.Overloading.FindElement<br /> "foregroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "foregroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217."))<br /> obj,<br /> Data.GI.Base.Attributes.AttrSetTypeConstraint<br /> (Data.GI.Base.Overloading.FindElement<br /> "foregroundRgba"<br /> (Data.GI.Base.Overloading.AttributeList obj)<br /> (((('GHC.TypeLits.Text "Unknown attribute \8216"<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "foregroundRgba")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217 for object \8216")<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.ShowType obj)<br /> 'GHC.TypeLits.:<>: 'GHC.TypeLits.Text "\8217."))<br /> GI.Gdk.Structs.RGBA.RGBA) =><br /> Reactive.Banana.Common.Colour<br /> -> m [Data.GI.Base.Attributes.AttrOp obj tag]</span><br />
<br />Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com0tag:blogger.com,1999:blog-5975524006824862804.post-82589880807119934442018-03-24T11:29:00.000-07:002018-04-25T07:41:12.810-07:00Why I don’t believe in the Second Amendment (and neither do you or the Supreme Court)<br />
The second amendment says:<br />
<blockquote>
<i>A well regulated Militia, being necessary to the
security of a free State, the right of the people to keep and bear
Arms, shall not be infringed.</i></blockquote>
This consists of two parts, the preamble “<i>A well regulated
Militia, being necessary to the security of a free State”</i><span style="font-style: normal;">
and the effective part “</span><i>the right of the people to keep
and bear Arms, shall not be infringed”</i><span style="font-style: normal;">.</span><br />
<span style="font-style: normal;">The </span><span style="font-style: normal;">meaning</span><span style="font-style: normal;">
of the effective part is very straightforward: “arms” means any
kind of weapon. A bowie knife is “arms”, and during the Cold War
the superpowers engaged in “Strategic Arms Limitations Talks”
(SALT). Although military technology has changed beyond all
recognition, the </span><span style="font-style: normal;">use</span><span style="font-style: normal;">
of the word “arms” </span><span style="font-style: normal;">to
mean</span><span style="font-style: normal;"> any kind of weapon</span><span style="font-style: normal;">
has not changed since 1791 when the Bill of Rights was ratified. Had
</span><span style="font-style: normal;">James Madison and the other</span><span style="font-style: normal;">
framers of the constitution been shown the modern world they would
have had no hesitation in agreeing that any form of modern weaponry
</span><span style="font-style: normal;">from a bowie knife to a nuke</span><span style="font-style: normal;">
was included in the </span><span style="font-style: normal;">definition</span><span style="font-style: normal;">.</span><br />
<br />
<span style="font-style: normal;">So it would seem that anyone in
the United States of America has the </span><span style="font-style: normal;">constitutional</span><span style="font-style: normal;">
right to purchase, say, a shoulder launched surface to air missile,
and to bear that armament anywhere they wish, such as near the end of
a runway where large passenger jets are taking off. </span><span style="font-style: normal;">That
would not be a good idea</span><span style="font-style: normal;">. </span><span style="font-style: normal;">If
military weapons were widely available then </span><span style="font-style: normal;">they
would be used</span><span style="font-style: normal;"> for mass murder
of unarmed civilian targets. Stephen Paddock, the Las Vegas shooter,
seems to have set out to kill as many people as possible, and he
managed to kill 58. </span><span style="font-style: normal;">An</span><span style="font-style: normal;">
Airbus A-380 typically has a maximum capacity of 525 </span><span style="font-style: normal;">plus
crew, </span><span style="font-style: normal;">and if one were bought
down over the right bit of city </span><span style="font-style: normal;">then
the body count </span><span style="font-style: normal;">c</span><span style="font-style: normal;">ould
go into the thousands</span><span style="font-style: normal;">. </span><span style="font-style: normal;">G</span><span style="font-style: normal;">iven
the </span><span style="font-style: normal;">opportunity</span><span style="font-style: normal;">
</span><span style="font-style: normal;">Paddock</span><span style="font-style: normal;">
would </span><span style="font-style: normal;">surely</span><span style="font-style: normal;">
have </span><span style="font-style: normal;">done so</span><span style="font-style: normal;">,
</span><span style="font-style: normal;">and he is not </span><span style="font-style: normal;">unique</span><span style="font-style: normal;">.</span><br />
<h1 class="western">
<span style="font-style: normal;">T</span><span style="font-style: normal;">he
Courts</span></h1>
<span style="font-style: normal;">T</span><span style="font-style: normal;">oday
t</span><span style="font-style: normal;">he original </span><span style="font-style: normal;">intent</span><span style="font-style: normal;">
of the Second Amendment is dead as a matter of established law. </span><span style="font-style: normal;">It
is <a href="https://en.wikipedia.org/wiki/Title_II_weapons">not legal</a> for anyone within the United States to make or buy heavy
weapons except under the most stringent and careful of controls.</span><br />
<br />
<div style="font-style: normal;">
This would have surprised and
disappointed the framers. They had just fought a war in which
ordinary armed citizens had beaten off the professional British army.
The framers saw this as an important lesson; the liberty of their new
nation would be assured by having an armed citizenry who could be
relied upon to form a militia and fight against any future tyrant,
foreign or domestic. This is what the preamble part of the Second
Amendment is about.</div>
<br />
<span style="font-style: normal;">I haven’t been able to find any
case in which the constitutionality of this ban</span><span style="font-style: normal;">
on heavy weapons</span><span style="font-style: normal;"> has ever
been truly tested. However </span><span style="font-style: normal;">the
courts have found that bans on semi-automatic rifles and the carrying
of handguns in public are constitutional, and the Supreme Court has
declined to review these </span><span style="font-style: normal;">decisions</span><span style="font-style: normal;">.</span><br />
<br />
<span style="font-style: normal;">In the case of the <a href="https://www.nbcnews.com/news/us-news/assault-weapons-not-protected-second-amendment-federal-appeals-court-rules-n724106">Maryland ban on assault rifles</a>, the 4</span><sup><span style="font-style: normal;">th</span></sup><span style="font-style: normal;">
Circuit Court of Appeals decided 10-4 that the ban was
constitutional. The judgement </span><span style="font-style: normal;">explains
this with</span><span style="font-style: normal;"> “</span><i>Put
simply, we have no power to extend Second Amendment protections to
weapons of war</i><span style="font-style: normal;">”. The framers
of the constitution would have found that </span><span style="font-style: normal;">bizzare</span><span style="font-style: normal;">;
to them the Second Amendment </span><span style="font-style: normal;">was</span><span style="font-style: normal;">
precisely about weapons of war.</span><br />
<br />
<span style="font-style: normal;">O</span><span style="font-style: normal;">n
the other hand back in 1939 in the <a href="https://en.wikipedia.org/wiki/United_States_v._Miller">Miller case</a> the Supreme Court
decided that a sawn-off shotgun was not protected by the Second
Amendment because it was not potentially a military weapon, and this
is still a valid precedent </span><span style="font-style: normal;">(they
may have been wrong about the military use of sawn-off shotguns in
particular, but the principle still stands)</span><span style="font-style: normal;">.
So the 4</span><sup><span style="font-style: normal;">th</span></sup><span style="font-style: normal;">
Circuit has decided that military weapons are unprotected, and the
Supreme Court has decided that non-military weapons are unprotected. According to these precedents it would seem that </span><span style="font-style: normal;">Americans
have the constitutional right to keep and bear arms, except for
military weapons and non-military weapons.</span><br />
<h1 class="western" style="font-style: normal;">
Self Defense</h1>
<span style="font-style: normal;">The </span><span style="font-style: normal;">most
recent </span><span style="font-style: normal;">case on the Second
Amendment that reach</span><span style="font-style: normal;">ed</span><span style="font-style: normal;">
the Supreme Court was </span><a href="https://en.wikipedia.org/wiki/District_of_Columbia_v._Heller"><i>District of Columbia v Heller</i></a><span style="font-style: normal;">
</span><span style="font-style: normal;">in 2008</span><span style="font-style: normal;">,
which was about the constitutionality of a ban on handguns and a
requirement that larger guns be kept unloaded and locked. The </span><span style="font-style: normal;">C</span><span style="font-style: normal;">ourt
found 5-4 that this ban violated the Second Amendment.</span><br />
<br />
<span style="font-style: normal;">Th</span><span style="font-style: normal;">e
Heller case</span><span style="font-style: normal;"> was about
restrictions on small arms rather than heavy weapons, and the
decision is rightly focussed on that. So the question of whether it
is constitutional to own heavy weapons was not directly considered.
However if the Court had found that small arms are protected by the
plain meaning of the Second Amendment, then someone would have used
this to challenge laws banning heavy weapons on the grounds that they
are equally protected by the Second Amendment. So to overturn the
handgun ban the Court had to </span><span style="font-style: normal;">do
two things. First,</span><span style="font-style: normal;"> </span><span style="font-style: normal;">they
had to ignore the plain meaning of the words they were analysing, </span><span style="font-style: normal;">and
second they had to find </span><span style="font-style: normal;">some
other</span><span style="font-style: normal;"> right to keep small
arms.</span><br />
<span style="font-style: normal;">T</span><span style="font-style: normal;">hey
</span><span style="font-style: normal;">managed the first part</span><span style="font-style: normal;">
by finding that the Second Amendment is not a</span><span style="font-style: normal;">n
unlimited right</span><span style="font-style: normal;">. They dispose of this in one brief paragraph:</span><br />
<br />
<blockquote class="tr_bq">
<span style="font-style: normal;"><i>Like most rights, the Second Amendment right is not unlimited. It is not
a right to keep and carry any weapon whatsoever in any manner
whatsoever and for whatever purpose: For example, concealed weapons
prohibitions have been upheld under the Amendment or state analogues.
The Court’s opinion should not be taken to cast doubt on longstanding
prohibitions on the possession of firearms by felons and the mentally
ill, or laws forbidding the carrying of firearms in sensitive places
such as schools and government buildings, or laws imposing conditions
and qualifications on the commercial sale of arms. Miller’s holding that
the sorts of weapons protected are those “in common use at the time”
finds support in the historical tradition of prohibiting the carrying of
dangerous and unusual weapons.</i> </span></blockquote>
<br />
<span style="font-style: normal;">It is true that, as with
other parts of the Bill of Rights, the government can restrict the
scope of a right where it has a pressing need to do so. Thus, for
instance the First Amendment has at various times been limited to exclude </span><span style="font-style: normal;">the
publication of </span><span style="font-style: normal;">libel,
copyright infringement, </span><span style="font-style: normal;">certain
</span><span style="font-style: normal;">secrets,</span><span style="font-style: normal;">
and </span><span style="font-style: normal;">various kinds of</span><span style="font-style: normal;">
pornography. However the scope </span><span style="font-style: normal;">of
First Amendment restrictions has only rarely been allowed to extend
beyond those areas. The only significant case </span><span style="font-style: normal;">in
modern times </span><span style="font-style: normal;">was the <a href="https://en.wikipedia.org/wiki/Sedition_Act_of_1918">Sedition Act of 1918</a>, which the Supreme Court found constitutional the
following year. However this is now considered to be a mistake; today
the heart of the First Amendment is seen as political speech and any
speech touching on politics enjoys the highest level of
constitutional protection. </span><span style="font-style: normal;">T</span><span style="font-style: normal;">he
framers of the Constitution would surely agree that this is the
proper state of affairs.</span><br />
<br />
<span style="font-style: normal;">However the restrictions on the
</span><span style="font-style: normal;">right to keep and bear arms</span><span style="font-style: normal;">
are not so limited. </span><span style="font-style: normal;">The
Second Amendment is primarily about military weapons, but u</span><span style="font-style: normal;">nder
</span><span style="font-style: normal;">federal </span><span style="font-style: normal;">law</span><span style="font-style: normal;">
</span><span style="font-style: normal;">today </span><span style="font-style: normal;">only
weapons of little military use </span><span style="font-style: normal;">may
be legally held by civilians</span><span style="font-style: normal;">.
Imagine if the First Amendment were restricted to stating opinions that are in "<a href="http://common use">common use</a>", with citizens only allowed to comment on local politics and </span><span style="font-style: normal;">speech</span><span style="font-style: normal;">
on </span><span style="font-style: normal;">state and </span><span style="font-style: normal;">national
affairs being restricted to those government officials employed to do
so. That would not be </span><span style="font-style: normal;">freedom
of speech as it is understood in most of the world; indeed it would
</span><span style="font-style: normal;">resemble</span><span style="font-style: normal;">
</span><span style="font-style: normal;">many dicatorships.</span><br />
<br />
<span style="font-style: normal;">So having found that the Second
Amendment does not in fact protect the right to keep and bear </span><span style="font-style: normal;">most
arms</span><span style="font-style: normal;">, the Supreme Court </span><span style="font-style: normal;">in
2008</span><span style="font-style: normal;"> had to find a right that
protected handguns. </span><span style="font-style: normal;">The</span><span style="font-style: normal;">y</span><span style="font-style: normal;">
found this in the right to self defense.</span><br />
<br />
<span style="font-style: normal;">Unlike the right to keep and bear
arms, the right to self defense is not enumerated in the Bill of
Rights. This is not a problem because the <a href="https://en.wikipedia.org/wiki/Ninth_Amendment_to_the_United_States_Constitution">Ninth Amendment</a>
specifically says that </span><span style="font-style: normal;">this
“</span><i>shall not be construed to deny or disparage others
retained by the people</i><span style="font-style: normal;">”.
</span><span style="font-style: normal;">A</span><span style="font-style: normal;">nyone
being attacked has the right to defend themselves with whatever means
may be at hand; if you are faced with an attacker then you may pick
up a golf club and swing at their head, or </span><span style="font-style: normal;">try
to stab them with a kitchen knife</span><span style="font-style: normal;">,
or just use your fists. The question in such cases is not whether the
weapon </span><span style="font-style: normal;">is</span><span style="font-style: normal;">
legal but whether </span><span style="font-style: normal;">it</span><span style="font-style: normal;">
</span><span style="font-style: normal;">is</span><span style="font-style: normal;">
used in a legitimate act of self defense. The same argument applies
to guns; if you are forced to defend yourself then using a gun is as
reasonable as using a knife, even if that gun was not one you were
legally permitted to have.</span><br />
<br />
<span style="font-style: normal;">However </span><span style="font-style: normal;">in
</span><i>Heller</i><span style="font-style: normal;"> </span><span style="font-style: normal;">the
Supreme Court, after exhaustive analysis of the history of self
defense law in both the United States and the United Kingdom,
concluded that the right to self defense included the right to keep a
hand gun </span><span style="font-style: normal;">in your house</span><span style="font-style: normal;">
for that purpose.</span><br />
<br />
<span style="font-style: normal;">A</span><span style="font-style: normal;">s
an aside, I should say that <a href="http://www2.law.ucla.edu/volokh/beararms/statecon.htm">some state constitutions</a> provide a right
to bear arms in self defense. Obviously in those states the situation
is different, but this post is about the US constitution.</span><br />
<br />
<div style="font-style: normal;">
I’m not going to go any further into
this question of personal self defense because this essay is not
about that; its about the Second Amendment. Heller is not about the
general right to keep and bear arms, its about the right to keep a hand gun
as self defense against home invasions. The fact that the Supreme
Court found it necessary to invoke a different right to protect hand
guns shows that they don’t believe in the Second Amendment either;
they actually upheld something different.</div>
<h1 class="western">
The NRA Version of the Second Amendment</h1>
<span style="font-style: normal;">T</span><span style="font-style: normal;">he
original purpose of the Second Amendment </span><span style="font-style: normal;">was
not individual defense against home invasion, it was </span><span style="font-style: normal;">collective
defense against tyranny. Despite a long consideration of the “</span><span style="font-style: normal;">well
regulated militia”</span><span style="font-style: normal;"> the
Heller decision is silent on this question.</span><br />
<br />
<span style="font-style: normal;">Supporters of the NRA are not
silent on this question; when asked to justify the right to keep and
bear small arms they claim that their guns are a bulwark against
</span><span style="font-style: normal;">government over-reach</span><span style="font-style: normal;">.
</span><a href="https://edition.cnn.com/2018/02/22/politics/wayne-lapierre-cpac-speech-nra/index.html"><span style="font-style: normal;">Some</span><span style="font-style: normal;">
</span></a><span style="font-style: normal;"><a href="https://edition.cnn.com/2018/02/22/politics/wayne-lapierre-cpac-speech-nra/index.html">go so far as to claim</a> that
overturning the Second Amendment is the first step in a deep and dark
plot to overthrow the rest of the Bill of Rights </span><span style="font-style: normal;">and
institute a socialist tyranny.</span><br />
<br />
<span style="font-style: normal;">However the historical facts do
not support the NRA rhetoric on this. Since 1776 the US government
has been responsible for a long and sad list of human rights
violations, </span><span style="font-style: normal;">but civilian
firearms have never prevented any of them</span><span style="font-style: normal;">.
The </span><span style="font-style: normal;">last</span><span style="font-style: normal;">
century </span><span style="font-style: normal;">alone saw</span><span style="font-style: normal;">
the following:</span><br />
<ul>
<li>
<div style="font-style: normal;">
The <a href="https://en.wikipedia.org/wiki/Sedition_Act_of_1918">Sedition Act of 1918</a>, which
directly violated the First Amendment by restricting political
speech.</div>
</li>
<li>
<div style="font-style: normal;">
The<a href="https://en.wikipedia.org/wiki/Internment_of_Japanese_Americans"> internment of Japanese Americans in World War II</a>, in violoation of pretty much every
constitutional right they had.</div>
</li>
<li>
<div style="font-style: normal;">
The “<a href="https://en.wikipedia.org/wiki/Jim_Crow_laws">Jim Crow</a>” laws of the
southern states, finally abolished in 1964. Not only were these laws
themselves an infringement on the constitutional rights of many citizens, but
they went alongside a grossly biased justice system that somehow
<a href="https://en.wikipedia.org/wiki/Tulsa_race_riot">never got around to prosecuting</a> white perpetrators of <a href="https://en.wikipedia.org/wiki/Lynching_in_the_United_States">violence against blacks</a>.</div>
</li>
<li>
<div style="font-style: normal;">
The Chicago “machine”
<a href="https://www.press.uillinois.edu/books/catalog/78ndq5ty9780252078552.html">rigging elections and protecting corrupt politicians</a>.</div>
</li>
<li>
<div style="font-style: normal;">
The FBI <a href="https://en.wikipedia.org/wiki/COINTELPRO">investigating and undermining</a> lawful civil rights movements under J. Edgar Hoover
</div>
</li>
<li>
<div style="font-style: normal;">
<a href="https://en.wikipedia.org/wiki/Guantanamo_Bay_detention_camp">Guantanamo Bay</a>.</div>
</li>
<li>
<span style="font-style: normal;">C</span><span style="font-style: normal;">ivil
forfeiture</span><span style="font-style: normal;"> used to</span><span style="font-style: normal;">
<a href="https://en.wikipedia.org/wiki/Civil_forfeiture_in_the_United_States">seize property</a> under mere suspicion.</span><br />
</li>
<li>
<a href="http://www.sdflaw.com/files/The_Law_of_Pretext_Stops_Since_Whren_v_United_States.pdf"><span style="font-style: normal;">P</span></a><span style="font-style: normal;"><a href="http://www.sdflaw.com/files/The_Law_of_Pretext_Stops_Since_Whren_v_United_States.pdf">retextual traffic stops</a> </span><span style="font-style: normal;">and “<a href="https://en.wikipedia.org/wiki/Parallel_construction">parallel construction</a>” </span><span style="font-style: normal;">used to
cover violations of the Fourth Amendment.</span><br />
</li>
<li>
<span style="font-style: normal;">A</span><span style="font-style: normal;">
national security apparatus that routinely <a href="https://www.theguardian.com/us-news/the-nsa-files">scans the email and other private communications of many American citizens</a> without any
effective oversight. </span>
<br />
</li>
</ul>
<span style="font-style: normal;">In all of this list civilian guns</span><span style="font-style: normal;">
</span><span style="font-style: normal;">were </span><span style="font-style: normal;">never
used to defend the rights of anyone. In the few cases where civilian
guns appeared, </span><span style="font-style: normal;">such as Jim
Crow lynch mobs,</span><span style="font-style: normal;"> they were </span><span style="font-style: normal;">on
the wrong side.</span><br />
<br />
<span style="font-style: normal;">Has there ever been a case wh</span><span style="font-style: normal;">ere</span><span style="font-style: normal;">
guns held by civilians were effective in preventing a violation of
constituional rights? I doubt it. The fact is that if the American
government decides to </span><span style="font-style: normal;">act</span><span style="font-style: normal;">
in violation of the constitution then there is nothing that a small
band of gun owners can do to stop them. </span><span style="font-style: normal;">T</span><span style="font-style: normal;">he
government can always bring in bigger guns and more of them. Various
small bands have <a href="https://en.wikipedia.org/wiki/Timeline_of_the_occupation_of_the_Malheur_National_Wildlife_Refuge">tried to defy</a> the government; it always ends badly. </span><span style="font-style: normal;">T</span><span style="font-style: normal;">oday
the only ways to change the way the </span><span style="font-style: normal;">American
</span><span style="font-style: normal;">government behaves are the
ballot box, the courts, and <strike>bribery</strike> <a href="https://en.wikipedia.org/wiki/Campaign_finance_in_the_United_States">making political donations</a>. Pointing guns
at government employees will <a href="https://xkcd.com/652/">simply get you shot</a>.</span><br />
<br />
<span style="font-style: normal;"><b>Edit: </b><a href="https://politics.stackexchange.com/questions/28162/when-was-the-second-amendment-last-used-to-fight-against-the-us-government-to-pr">This Stack Exchange question</a> points to the <a href="https://en.wikipedia.org/wiki/Wounded_Knee_incident">Wounded Knee incident</a> in 1973, which was indeed a case where civilians with guns prevented the infringement of constitutional rights by the government. Another answer to the same question also cited the <a href="https://en.wikipedia.org/wiki/Bundy_standoff">Bundy standoff</a> of 2014 and the related <a href="https://en.wikipedia.org/wiki/Occupation_of_the_Malheur_National_Wildlife_Refuge">Malheur occupation</a> of 2016, but they are not good examples. The Bundy standoff seems to have been driven by Cliven Bundy's fringe beliefs about federal land ownership rather than a real infringement of constitutional rights, and the Malheur occupation ended with the shooting of one participant and the arrest of everyone else.</span><br />
<h1 class="western">
The Second Amendment Today</h1>
The Bill of Rights was written at a time when there was little
difference between military and civilian firearms, military ships
were in civilian hands (“<a href="https://en.wikipedia.org/wiki/Privateer#United_States">privateers</a>”), and the fastest means of
communication was a rider on horseback. Firearms were mostly <a href="https://en.wikipedia.org/wiki/Musket">muzzle-loading flintlocks</a>; a skilled gunman might get off two or three
rounds per minute and for an individual they were effectively
single-shot weapons. They were also not very accurate. Effective use
of such weapons required collective action to keep the enemy at bay
while you reloaded. The only thing bigger than a rifle was a cannon,
which was also muzzle loading and inaccurate.<br />
<br />
Today the scope of military weapons has expanded along with the
rest of our technology. If the Second Amendment were taken at face
value today then many high explosive ranged weapons capable of major
destruction would be within the financial reach of much of the
population. I opened this essay by inviting you to consider a
shoulder launched surface to air missile. I’d like to close by
inviting you to consider a school shooter with a <a href="https://en.wikipedia.org/wiki/Bazooka">bazooka</a> firing a
<a href="https://en.wikipedia.org/wiki/White_phosphorus_munitions#Military_regulations">white phosphorus</a> munition.<br />
<br />
Most of the Bill of Rights has fared surprisingly well with time;
the First Amendment right to free speech now applies to <a href="https://en.wikipedia.org/wiki/Joseph_Burstyn,_Inc._v._Wilson">film</a>, <a href="https://law.justia.com/constitution/us/amendment-01/15-cable-television.html">television</a>, <a href="https://www.comparitech.com/blog/vpn-privacy/the-first-amendment-what-it-means-free-speech-online/">web sites</a> and
<a href="https://www.eff.org/deeplinks/2015/04/remembering-case-established-code-speech">computer software</a>, but the basic principle is still the same. This is equally true
of Amendments 3 to 10. The Second stands out as a product of its
time, overtaken by changes in technology and society.<br />
<br />
If you truly believe in the Second Amendment then you believe that
any military weapon held by the armed forces of the United States
should also be legally available to most civilians. I think the
number of people who really believe that is very small. Even the
National Rifle Association only believes in the right to keep rifles.
There is no National Bazooka Association (actually that’s <a href="https://www.facebook.com/nationalbazookaassociation/">not true</a>,
but it seems to be only one person and may just be a joke).<br />
<br />
<br />
<br />
<style type="text/css">h1 { margin-bottom: 0.08in; }h1.western { font-family: "Liberation Sans", sans-serif; font-size: 18pt; }h1.cjk { font-family: "Source Han Sans CN Regular"; font-size: 18pt; }h1.ctl { font-family: "Lohit Devanagari"; font-size: 18pt; }p { margin-bottom: 0.1in; line-height: 120%; }</style>Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com27tag:blogger.com,1999:blog-5975524006824862804.post-5951333970940223982018-02-25T09:06:00.000-08:002018-02-25T09:39:16.988-08:00Haskell with Reactive Banana and GTK3<p>
I've been doing some GUI coding recently using a combination of Reactive Banana and GTK3. I started out with just <a href="https://hackage.haskell.org/package/gtk3">GTK3</a>, but I could see it wasn't going to scale because everything GTK3 does is in the IO monad. I found I was having to create <font face="monospace">IORef</font>s to track the application state, and then pass these around for reading and writing by various event handlers. While the application was small this was manageable, but I could see that it was going to grow into a pile of imperative spaghetti as time went on.</p>
<p>
I knew about <a href="https://en.wikipedia.org/wiki/Functional_reactive_programming">functional
reactive programming</a> (FRP), and went on the hunt for a framework
that would work with GTK3. I chose
<a href="https://hackage.haskell.org/package/reactive-banana">Reactive
Banana</a> despite the silly name because it seemed to be targeted at
desktop GUI applications rather than games and simulations.</p>
<h2 class="western">
Connecting to GTK3</h2>
<p>
FRP is based around two key abstractions:</p>
<ul> <li/>
<p style="margin-bottom: 0in">
Events are instants in time that
carry some data. When the user clicks on a button in a GUI you want
an event to denote the fact that the button was clicked. If the user
moves a slider then you want an event with the new position of the
slider.
</p>
<li/>
<p>
Behaviors carry data that changes over time. In theory this
change can be continuous; for instance if you simulate a bouncing
ball then the position of the ball is a behavior; at any point in
time you can query it, and queries at different times will get
different answers. However Reactive Banana only supports behaviors
that change in response to events and remain constant the rest of
the time. Thats fine: my application responds to user events and
doesn't need continuous changes. Take the slider event I mentioned
above: when the user moves the slider you want to update a
<font face="monospace">sliderPosition</font>
behavior with the latest position so that other parts of the program
can then use the value later on.
</p>
</ul>
<p>
In Reactive Banana you can convert an event into a behavior with the "<font face="monospace">stepper</font>" function. You can also get an event back when a behavior changes </p>
<p>
Behaviors are a lot like the <font face="monospace">IORef</font>s I was already using, but events are what make the whole thing scalable. In a large application you may want several things to happen when the user clicks something, but without the Event abstraction all of those things have to directly associated. This harms modularity because it creates dependencies between modules that own callbacks and modules that own widgets, and also causes uncertainty within callbacks about which other callbacks might already have been invoked. With FRP the widget creator can just return an Event without needing to know who receives it, and behaviours are not updated until all events have been executed.</p>
<p>
There is already a <a href="https://hackage.haskell.org/package/reactive-banana-wx">binding between Reactive Banana and WxHaskell</a>, but nothing for GTK. So my first job was to figure this out. Fortunately it turned out to be very simple. Every widget in GTK3 has three key lists of things in its API:</p>
<ul> <li/>
<p>
IO functions. These are used to create widgets, and also to get and set various parameters. So for instance the slider widget has functions like this (I'm glossing over some typeclass stuff here. For now just take it that a slider has type <font face="monospace">Range</font>):
<font color="#0000ff" face="monospace"><pre> rangeGetValue :: Range -> IO Double
rangeSetValue :: Range -> Double -> IO ()</pre>
</font></p>
<li/>
<p>
Attributes. These are kind of like <a href="https://hackage.haskell.org/package/lens">lenses</a> on the widget, in that they let you both read and write a value. However unlike Haskell lenses this only works in the IO monad. So for instance the slider widget has an attribute:
</br>
<font color="#0000ff" face="monospace"><pre> rangeValue :: Attr Range Double</pre>
</font></p>
<p>
You can access the attributes of a widget using the <a href="https://hackage.haskell.org/package/glib-0.13.5.0/docs/System-Glib-Attributes.html"><font face="monospace">get</font> and <font face="monospace">set</font></a> functions. This is equivalent to using the two IO functions above.</p>
<li/>
<p>
Signals. These are hooks where you can attach a callback to a widget using the
<a href="https://hackage.haskell.org/package/glib-0.13.5.0/docs/System-Glib-Signals.html">
<font face="monospace">on</font></a>
function. A callback is an IO monad action which is invoked
whenever the signal is triggered. This is usually when the user does
something, but can also be when the program does something. For
instance the slider widget has a signal</p>
<font color="#0000ff" face="monospace"><pre> valueChanged :: Signal Range (IO ())</pre>
</font>
The last argument
is the type of the callback. In this case it takes no parameters and
returns no value, so you can hook into it like this:
<font color="#0000ff" face="monospace"><pre> on mySlider valueChanged $ do
v <- rangeGetValue mySlider</pre>
</font>
</ul>
<p>
One subtlety about GTK3 signals is that they are often only triggered when the underlying value actually changes, rather than every time the underlying setter function is called. So if the slider is on 10 and you call "<font face="monospace">rangeSetValue 9</font>" then the callback is triggered in exactly the same way as when the user moves it. However if you call "<font face="monospace">rangeSetValue 10</font>" then the callback is not triggered. This lets you cross-connect widgets without creating endless loops.</p>
<h3>
Connecting GUI Inputs</h3>
<p>
The crucial thing is that GTK signals and attributes are isomorphic with Reactive Banana events and behaviors. So the following code gets you quite a long way:</p>
<font color="#0000ff" face="monospace"><pre>
registerIOSignal :: (MonadIO m) =>
object
-> Signal object (m a)
-> m (a, b)
-> MomentIO (Event b)
registerIOSignal obj sig act = do
(event, runHandlers)
liftIO $ obj `on` sig $ do
(r, v) <- act
liftIO $ runHandlers v
return r
return event</pre>
</font> <p>
There are a few wrinkles that this has to cope with:</p>
<p>
First, a few signal handlers expect the callback to return something other than "<font face="monospace">()</font>". Hence the "<font "monospace">a</font>" type parameter above.</p>
<p>
Second, the callback doesn't usually get any arguments, such as the current slider position. Its up to the callback itself to get whatever information it needs. Hence you still need to write some callback code.</p>
<p>
Third, some signals work in monads other than just "IO". Usually these are of the form "<font face="monospace">ReaderT IO</font>" (that is, IO plus some read-only context). The "<font face="monospace">m</font>" type parameter allows for this.</p>
<p>
So now we can get a Reactive Banana event for the slider like this:</p>
<font color="#0000ff" face="monospace"><pre> sliderEvent <- registerIOSignal mySlider valueChanged $ do
v <- rangeGetValue mySlider
return ((), v)</pre>
</font> <p>
The two values in the "<font face="monospace">return</font>" are the return value for the callback (which is just <font face="monospace">()</font> in this case) and the value we want to send out in the Event.</p>
<p>
Some signals do provide parameters directly to the callback, so you need a family of functions like this:</p>
<font color="#0000ff" face="monospace"><pre> registerIOSignal1 :: (MonadIO m) =>
object
-> Signal object (a -> m b)
-> (a -> m (b, c))
-> MomentIO (Event c)
registerIOSignal2 :: (MonadIO m) =>
object
-> Signal object (a -> b -> m c)
-> (a -> b -> m (c, d))
-> MomentIO (Event d)</pre>
</font> <p>
And so on up to <font face="monospace">registerIOSignal4</font>, which is the longest one I have needed so far.</p>
<h3>
Connecting Outputs</h3>
<p>
Outputs are simpler than inputs. Reactive Banana provides a function for linking an event to an IO action:</p>
<font color="#0000ff" face="monospace"><pre> reactimate :: Event (IO ()) -> MomentIO ()</pre>
</font> <p>
This takes an event carrying IO actions and executes those actions as they arrive. The "<font face="monospace">MomentIO</font>" return value is the monad used for building up networks of events and behaviors: more of that in "Plumbing" below.</p>
<p>
Events are functors, so the usual pattern for using reactimate looks like this:</p>
<font color="#0000ff" face="monospace"><pre> reportThis :: Event String -> MomentIO ()
reportThis ev = do
let ioEvent = fmap putStrLn ev
reactimate ioEvent</pre>
</font> <p>
The argument is an event carrying a string. This is converted into an event carrying IO actions using "fmap", and the result is then passed to reactimate. Obviously this can be reduced to a single line but I've split it out here to make things clearer.</p>
<p>
So we can link an event to a GTK attribute like this:</p>
<font color="#0000ff" face="monospace"><pre> eventLink :: object -> Attr object a -> Event a -> MomentIO ()
eventLink obj attr =
reactimate . fmap (\v -> set obj [attr := v])</pre>
</font> <p>
Whenever the argument event fires the attribute will be updated with the value carried by the event.</p>
<p>
Behaviors can be linked in the same way. Reactive Banana provides the "<font face="monospace">changes</font>" function to get an event whenever a behavior might have changed. However this doesn't quite work the way you would expect. The type is:</p>
<font color="#0000ff" face="monospace"><pre> changes :: Behavior a -> MomentIO (Event (Future a))</pre>
</font> <p>
The "<font face="monospace">Future</font>" type reflects the fact that a behavior only changes <i>after</i> the event processing has finished. This lets you write code that cross-links events and behaviors without creating endless loops, but it means you have to be careful when accessing the current value of a behavior. More about this in "Plumbing" below.</p>
<p>
To cope with these "<font face="monospace">Future</font>" values there is a special version of "<font face="monospace">reactimate</font>" called "<font face="monospace">reactimate'</font> " (note the tick mark). You use it like this:</p>
<font color="#0000ff" face="monospace"><pre> behaviorLink :: object -> Attr object a -> Behavior a -> MomentIO ()
behaviorLink obj attr bhv = do
fe <- changes bhv
reactimate' $ fmap (fmap (\v -> set obj [attr := v])) fe</pre>
</font> <p>
This will update the attribute whenever an event occurs which feeds in to the behavior. Note that this will still happen even if the new value is the same as the old; unlike GTK the Reactive Banana library doesn't cancel updates if the new and old values are the same.</p>
<h2>
Plumbing</h2>
<h3>
The Basic Concepts</h3>
<p>
Reactive Banana events and behaviors are connected together in the <font face="monospace">MomentIO</font> monad. This is an instance of <a href="https://hackage.haskell.org/package/base-4.10.1.0/docs/Control-Monad-Fix.html"> <font face="monospace">MonadFix</font></a> so you can use <a href="https://ocharles.org.uk/blog/posts/2014-12-09-recursive-do.html"> recursive do</a> notation, letting you create feedback loops between behaviors and events. <font face="monospace">MomentIO</font> is also an instance of <font face="monospace">MonadIO</font>, so you can use <font face="monospace">liftIO</font> to bring GTK widget actions into it.</p>
<p>
To set up a dialog containing a bunch of GTK widgets you do the following things in the <font face="monospace">MomentIO</font> monad:</p>
<ul> <li/>
<p style="margin-bottom: 0in">
Use <font face="monospace">liftIO
</font>on GTK functions to set up the widgets and
arrange them in layout boxes in the same way you would if you were
just using bare GTK.
</p>
<li/>
<p style="margin-bottom: 0in">
Use <font face="monospace">registerIOSignal</font>
to get Reactive Banana events from the widgets.
</p>
<li/>
<p style="margin-bottom: 0in">
Use the Reactive Banana
combinators to create new events and behaviors reflecting the
application logic you want.
</p>
<li/>
<p>
Use <font face="monospace">eventLink</font>
and <font face="monospace">behaviorLink</font>
to update widget attributes.
</p>
</ul>
<p>
For instance you can have a pop-up dialog containing a bunch of input widgets with events attached to their values. Lets say these fields are arguments to the "<font face="monospace">FooData</font>" constructor, and you also have a function "<font face="monospace">fooValid :: FooData -> Bool</font>". You can then write your code like this:</p>
<font color="#0000ff" face="monospace"><pre> fooDialog :: FooData -> MomentIO (Widget, Event FooData)
fooDialog (FooData v1 v2)= do
-- Create GTK widgets w1, w2 and okButton.
-- Put them all in a top-level "dialog" container.
.... your GTK code here.
-- Connect events ev1, ev2 to the values of w1 and w2.
.... your calls to registerIOSignal here.
okClick <- registerIOSignal okButton buttonActivate $ return ((), ())
bhv1 <- stepper v1 ev1
bhv2 <- stepper v2 ev2
let
fooB = FooData <$> bhv1 <*> bhv2
-- Behavior is an Applicative, so fooB :: Behavior FooData
validInput = fooValid <$> fooB
result = const <$> fooB <@> okClick
behaviorLink okButton widgetSensitive validInput
return (dialog, result)</pre>
</font> <p>
The last line but one links the "<font face="monospace">validInput</font>" behavior to the OK button sensitivity. So if the input data is not valid then the OK button is greyed out and will not respond to clicks. You can use the same technique do other more informative things like highlighting the offending widget or displaying a helpful message in another widget.</p>
<p>
The "<font face="monospace">result =</font>" line needs a bit of explanation. The "<font face="monospace"><@></font>" combinator in Reactive Banana works like the applicative "<font face="monospace"><*></font>" except that its second argument is an event rather than a behavior. The result is an event that combines the current value of the "<font face="monospace">fooB</font>" behavior with the value from the "<font face="monospace">okClick</font>" event. In this case the button click carries no information, so we use the function "<font face="monospace">const</font>" to just take the current behavior value.</p>
<p>
One tip: when writing pure functions that are going to be called using the "<font face="monospace"><@></font>" combinator its a good idea to put the argument that will come from the event last.</p>
<h3>
Keeping it Modular</h3>
<p>
I have found that these are good rules for designing applications in Reactive Banana:</p>
<ul> <li/>
<p>
Functions in the MomentIO monad
should take events and behaviors as parameters, but only return
events.</p>
<li/>
<p>
Avoid returning a behavior
unless you are sure that this is the only function that needs to
change it.</p>
<li/>
<p>
Keep the behavior definitions at the top of the call stack
where they are used. If in doubt, defer the job of defining the
behavior to your caller.
</p>
</ul>
<p>
This lets you write independent modules that all have a say in updates to some shared value. The shared value should be represented as a Behavior, and updates to it as Events which either carry new values or (better) update functions. So you have a bunch of independent editor functions and a top level function which looks like this:</p>
<font color="#0000ff" face="monospace"><pre> editor1, editor2, editor3 ::
Behavior FooData -> MomentIO (Event (FooData -> FooData))
fooDataManager :: FooData -> MomentIO ()
fooDataManager start = mdo -- Recursive do notation.
edit1 <- editor1 fooB
edit2 <- editor2 fooB
edit3 <- editor3 fooB
fooB <- accumB start $ unions [edit1, edit2, edit3]
-- accumB applies each event function to the accumulated behaviour value. </pre>
</font> Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com1tag:blogger.com,1999:blog-5975524006824862804.post-34192163681804521762016-12-03T15:04:00.002-08:002016-12-03T15:12:20.907-08:00What duties to software developers owe to users?<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
I was reading <a href="https://medium.freecodecamp.com/the-code-im-still-ashamed-of-e4c021dff55e">this blog post</a>, entitled "The code I’m still ashamed of". </div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
<br /></div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
TL; DR: back in 2000 the poster, Bill Sourour, was employed to write a web questionnaire aimed at teenage girls that purported to advise the user about their need for a particular drug. In reality unless you said you were allergic to it, the questionnaire always concluded that the user needed the drug. Shortly after, Sourour read about a teenage girl who had possibly committed suicide due to side effects of this drug. He is still troubled by this.</div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
<br /></div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
Nothing the poster or his employer did was illegal. It may not even have been unethical, depending on exactly which set of professional ethics you subscribe to. But it seems clear to me that there is something wrong in a program that purports to provide impartial advice while actually trying to trick you into buying medication you don't need. Bill Sourour clearly agrees.</div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
<br /></div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
Out in meatspace we have a clearly defined set of rules for this kind of situation. Details vary between countries, but if you consult someone about legal, financial or medical matters then they are generally held to have a "<a href="https://www.law.cornell.edu/wex/fiduciary_duty">fiduciary duty</a>" to you. The term derives from the Latin for "faithful". If X has a fiduciary duty to Y, then X is bound at all times to act in the best interests of Y. In such a case X is said to be "the fiduciary" while Y is the "beneficiary".</div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
<br /></div>
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
In many cases fiduciary duties arise in clearly defined contexts and have clear bodies of law or other rules associated with them. If you are the director of a company then you have a fiduciary duty to the shareholders, and most jurisdictions have a specific law for that case. But courts can also find fiduciary duties in other circumstances. In English law the general principle <a href="https://en.wikipedia.org/wiki/Fiduciary">is as follows</a>:</div>
<blockquote class="tr_bq">
<div class="graf graf--h3 graf--leading graf--title" id="74c6" name="74c6">
"<i>A fiduciary is someone who has undertaken to act for and on behalf of
another in a particular matter in circumstances which give rise to a
relationship of trust and confidence.</i>"</div>
</blockquote>
It seems clear to me that this describes precisely the relationship between a software developer and a user. The user is not in a position to create the program they require, so they use one developed by someone else. The program acts as directed by the developer, but on behalf of the user. The user has to trust that the program will do what it promises, and in many cases the program will have access to confidential information which could be disclosed to others against the user's wishes.<br />
<br />
These are not theoretical concerns. "Malware" is a very common category of software, <a href="https://en.wikipedia.org/wiki/Malware">defined as</a>:<br />
<blockquote class="tr_bq">
<i>any software used to disrupt computer or mobile operations, gather
sensitive information, gain access to private computer systems, or
display unwanted advertising. </i></blockquote>
Sometimes malware is illicitly introduced by hacking, but in many cases the user is induced to run the malware by promises that it will do something that the user wants. In that case, software that acts against the interests of the user is an abuse of the trust placed in the developer by the user. In particular, the potential for software to "gather sensitive information" and "gain access to private computer systems" clearly shows that the user must have a "relationship of trust and confidence" with the developer, even if they have never met.<br />
<br />
One argument against my thesis came up when I <a href="http://law.stackexchange.com/questions/14610/does-a-software-developer-have-a-fiduciary-duty-to-the-user">posted a question</a> about this to Legal forum on Stack Exchange. The answer I got from Dale M argued that:<br />
<blockquote class="tr_bq">
<br />
Engineers (including software engineers) do not have this [relationship of confidence] and AFAIK a
fiduciary duty between an engineer and their client has never been
found, even where the work is a one-on-one commission.</blockquote>
I agree that, unlike a software developer, all current examples of a fiduciary duty involve a relationship in which the fiduciary is acting directly. The fiduciary has immediate knowledge of the circumstances of the particular beneficiary, and decides from moment to moment to take actions that may or may not be in the beneficiary's best interest. In contrast a software developer is separated in time from the user, and may have little or no knowledge of the user's situation.<br />
<br />
I didn't argue with Dale M because Stack Exchange is for questions and answers, not debates. However I don't think that the distinction drawn by Dale M holds for software. An engineer designing a bridge is not in a position to learn the private information of those who cross the bridge, but a software engineer is often in a position to learn a great deal about the users of their product. It seems to me that this leads inescapably to the conclusion that software engineers do have a relationship of confidence with the user, and that this therefore creates a fiduciary duty.<br />
<br />
Of course, as Dale M points out, nobody has ever persuaded a judge that software developers owe a fiduciary duty, and its likely that in practice its going to be a hard sell. But to go back to the example at the top, I think that Bill Sourer, or his employer, did owe a fiduciary duty to those people who ran the questionnaire software he wrote, because they disclosed private information in the expectation of getting honest advice, and the fact that they disclosed it to a program instead of a human makes no difference at all.<br />
<br />
<br />
<h4>
Addendum: Scope of duty </h4>
This section looks at exactly what the scope of the fiduciary duty is. It doesn't fit within the main text of this essay, so I've put it here.<br />
<br />
Fortunately there is no need for a change in the law regarding fiduciary
duty. The existence of a fiduciary duty is based on the nature of the
relationship between principal and agent, although in some countries
specific cases such as company directors are covered by more detailed
laws. <br />
<br />
First it is necessary to determine exactly who the fiduciary is. So far I have talked about "the software developer", but in practice software is rarely written by a single individual. We have to look at the authority that is directing the effort and deciding what functions will be implemented. If the software is produced by a company then treating the company as the fiduciary would seem to be the best approach, although it might be more appropriate to hold a senior manager liable if they have exceeded their authority.<br />
<br />
As for the scope, I'm going to consider the scope of the fiduciary duty imposed on company directors and consider whether an analogous duty should apply to a software developer:<br />
<br />
<ul>
<li>Duty of care: for directors this is the duty to inform themselves and take due thought before making a decision. One might argue that a software developer should have a similar duty of care when writing software, but this is already handled through normal negligence. Elevating the application of normal professional skill to a fiduciary duty is not going to make life better for the users. However there is one area where this might be applied: lack of motive to produce secure software is <a href="http://www.theregister.co.uk/2016/11/16/experts_to_congress_you_must_act_on_iot_security_congress_encourage_industry_to_develop_best_practices_you_say/?mt=1479344754858">widely recognised</a> as a significant problem, and is also an area where the "confidence" aspect of fiduciary duty overlaps with a duty of care. Therefore developers who negligently fail to consider security aspects of their software should be considered to have failed in their fiduciary duty.</li>
<li>Duty of loyalty: for directors this is the duty not to use their position to further their private interests. For a software developer this is straightforward: the developer should not use their privileged access to the user's computer to further their private interests. So downloading information from the users computer (unless the user explicitly instructs this to happen) should be a breach of fiduciary duty. So would using the processing power or bandwidth owned by the user for the developers own purposes, for instance by mining bitcoins or sending spam.</li>
<li>Duty of good faith: the developer should write code that will advance the user's interests and act in accordance with the user's wishes at all times.</li>
<li>Duty of confidentiality: if the developer is entrusted with user information, for example because the software interfaces with cloud storage, then this should be held as confidential and not disclosed for the developer's benefit.</li>
<li>Duty of prudence: This does not map onto software development.</li>
<li>Duty of disclosure: for a director this providing all relevant information to the shareholders. For a software developer, it means completely and honestly documenting what the software does, and particularly drawing attention to any features which a user might reasonably consider against their interests. Merely putting some general clauses in the license is not sufficient; anything that could reasonably be considered to be contrary to the user's interests should be prominently indicated in a way that enables the user to prevent it.</li>
</ul>
One gray area in this is software that is provided in exchange for personal data. Many "free" apps are paid for by advertisers who, in addition to the opportunity to advertise to the user, also pay for data about the users. On one hand, this involves the uploading of personal data that the user may not wish to share, but on the other hand it is done as part of an exchange that the user may be happy with. This comes under the duty of disclosure. The software should inform the user that personal data will be uploaded, and should also provide a detailed log of exactly what has been sent. Thus users can make informed decisions about the value of the information they are sending, and possibly alter their behavior when they know it is being monitored. <br />
<ul>
</ul>
<br />
<br />Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com2tag:blogger.com,1999:blog-5975524006824862804.post-23187605810181765002016-03-14T14:32:00.000-07:002016-03-14T14:32:11.892-07:00Letter to my MP about the Investigatory Powers BillI've just sent this email to my MP. Hopefully it will make a difference. I've asked for permission to post her reply.<br />
<br />
---------------------------<br />
<br />
Dear Ms Fernandes,
<br />
<br />I am a resident of [redacted]. My address is [redacted]. I am writing to you a second time about the proposed
Investigatory Powers Bill. I wrote to you about this on 5th November
2015 urging you to try to mitigate the worst aspects of this bill, and
now I am writing to urge you to vote against this bill when it comes to
Parliament.
<br />
<br />I am deeply concerned about the powers that this bill would give to the
Home Secretary. However in order to keep this email reasonably short I
will concentrate on one particularly dangerous power.
<br />
<br />If this bill becomes law then the Home Secretary would be able to order
any "communications company" (the term could mean anyone involved in
providing software or equipment that enables communication) to install
any surveillance feature the Home Secretary wishes. The recipient of
this order would be unable to appeal against it, and would be prevented
from revealing the existence of the order. There is no sunset time on
this gag clause: it will last as long as the Home Secretary and the
security services wish to maintain it.
<br />
<br />It is true that these orders will also have to be signed off by a judge,
but that will only verify that the order complies with whatever
procedures are in place at the time. Furthermore these judges will only
ever hear one point of view on the reasonableness and proportionality of
the orders, and this can only result in the erosion of these safeguards
over time.
<br />
<br />
<br />I want to illustrate the danger of this power to weaken security by
showing how it would impact a common method of selecting encryption keys
called Diffie-Hellman Key Exchange. This method is used by web browsers
and email programs whenever they make a secure connection (e.g. to web
addresses starting "https"). It is also used by "Virtual Private
Networks" (VPNs) which are widely used by businesses to allow employees
to work remotely, and I expect that Parliament has one to allow MPs to
access their email. You may even be using it to read this.
<br />
<br />I want to show that any attempt to intercept messages where
Diffie-Hellman is used will greatly weaken it, and that this will worsen
our security rather than improving it. I will show this by linking the
NSA to the compromise of the Office of Personnel Management (OPM) in
America last year.
<br />
<br />I don't propose to explain the technical details of Diffie-Hellman. What
it means is that two computers can exchange a few messages containing
large random numbers, and at the end of this they will share a secret
key without that key ever having been sent over the Internet.
<br />
<br />Suppose that a communications company provides software that uses
Diffie-Hellman, and receives an order from the Home Secretary that they
must make the encrypted messages available to law enforcement and the
intelligence agencies. What are they to do? They never see the secret
keys, so they must do one of the following:
<br />
<br />1: Modify the software to send a copy of the chosen key to someone. This
is far less secure, and also very obvious. Anyone monitoring the packets
sent by the programs will instantly see it.
<br />
<br />2: Modify the software to make the keys or the encryption weak in a
non-obvious way so that the UK intelligence agencies can determine what
the key is. For instance, the random numbers might be made more
predictable in a subtle way.
<br />
<br />These are the only two ways in which the communications company can
comply with the order.
<br />
<br />We have seen what happens when Option 2 is chosen, because this was done
to Juniper Networks firewall product [see ref 1 below]. Someone deliberately inserted
"unauthorised code" which weakened the encryption used by this product
in a very specific and deliberate way. There is no possibility that this
was an accidental bug. The responsible party is widely believed to be
the NSA, because secret briefings released by Edward Snowden made
reference to the ability to intercept data sent via this product [ref 2],
and it would be much easier for the NSA to infiltrate an American
company than for anyone else to do it.
<br />
<br />However there is something important that happens when software is
updated: hackers (including foreign governments) scrutinize the updates
to see what has changed. Normally they find that the old version of the
software had a security hole which is now patched, so the patch flags up
a way to attack computers that haven't been updated yet. But in this
case when Juniper issued an update to their firewall software these
hackers found the security hole in the <b class="moz-txt-star"><span class="moz-txt-tag">*</span>new<span class="moz-txt-tag">*</span></b> software.
<br />
<br />Doing this kind of analysis in a systematic way for many security
products is a very large job. Doing it in secret requires the resources
of a government. So now not only could the NSA intercept communications
sent via Juniper firewalls, but so could an unknown number of foreign
governments. The Chinese were almost certainly one of them. Other
nations known to have invested in cyber-attack capabilities include the
Russia, Israel and North Korea (although the last is probably not as
capable yet).
<br />
<br />Juniper products are widely used by the US Government. This is likely to
have been one of the ways in which the Office of Personnel Management
(OPM) was penetrated last year [ref 3]. The Chinese government is the prime
suspect in this hack, through which the attackers have obtained copies
of the security clearance applications of everyone who has ever worked
for the US government.
<br />
<br />So it seems that the NSA, by introducing a supposedly secret "back door"
into a widely used product, cleared the way for the Chinese to obtain
secret files on everyone who has ever worked for their government,
including all of their legislators and everyone who works at the NSA.
Nice job breaking it, Hero!
<br />
<br />
<br />Now it is true that this is circumstantial; we have no hard evidence
that the Juniper back door was inserted by the NSA, no hard evidence
that the Chinese found it, and no hard evidence that this contributed to
the OPM hack. But each of these is a big possibility. Even if the OPM
hack didn't happen in exactly that way, deliberately weakening security
makes events like this much more likely. If the Home Secretary orders a
company to introduce weakened security, that fact will become apparent
to anyone with the resources to dig for it. Once armed with that fact,
they can attack through the same hole.
<br />
<br />Furthermore, we would never find out when a disaster like the OPM hack
happens under the regime described in the Investigatory Powers bill.
Suppose that, thanks to the weakened security ordered by the Home
Secretary, secret government files are obtained by a hostile power, and
the communications company executives are called before a Parliamentary
Inquiry to account for their negligence; how can they defend themselves
if they are legally prohibited from revealing their secret orders?
<br />
<br />More generally, we will never be allowed to learn about the negative
effects of these secret orders. It would embarrass those who issued
them, and they are exactly the people who would have to give permission
for publication. So if Parliament passes this bill it will never be
allowed to learn about the problems it causes, and hence never be able
to remedy the mistake.
<br />
<br />I have focused on only one of the measures in the Investigatory Powers
bill here, but there are many others in the bill that cause me great
concern. To go through the whole bill in this level of detail would make
this email far longer, and I know that you have many calls on your time.
I can only ask you to believe that there are many similar issues. For
these reasons I must urge you to vote against the bill when it reaches
the House of Commons.
<br />
<br />Yours sincerely,
<br />
<br />Paul Johnson.
<br />
<br />
<br />[1]
<a class="moz-txt-link-freetext" href="http://forums.juniper.net/t5/Security-Incident-Response/Important-Announcement-about-ScreenOS/ba-p/285554">http://forums.juniper.net/t5/Security-Incident-Response/Important-Announcement-about-ScreenOS/ba-p/285554</a>
<br />
<br />[2]
<a class="moz-txt-link-freetext" href="https://assets.documentcloud.org/documents/2653542/Juniper-Opportunity-Assessment-03FEB11-Redacted.txt">https://assets.documentcloud.org/documents/2653542/Juniper-Opportunity-Assessment-03FEB11-Redacted.txt</a>
<br />
<br />[3] <a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/Office_of_Personnel_Management_data_breach">https://en.wikipedia.org/wiki/Office_of_Personnel_Management_data_breach</a>
Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com1tag:blogger.com,1999:blog-5975524006824862804.post-7075155209909399862015-03-28T07:59:00.002-07:002015-04-20T10:43:56.406-07:00Google Maps on Android demands I let Google track me<b>Updated: see below.</b><br />
<br />
I recently upgraded to Android 5.1 on my Nexus 10. One app I often use is Google Maps. This has a "show my location" button:<br />
<img src="http://i.stack.imgur.com/7VDwn.png" height="30" id="irc_mi" width="30" /> <br />
When I clicked on this I got the following dialog box:<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSJ7KZiowmk7dTB2HYog23ED7ZD43B6-9jeraexWXn_DuFUX8BddqmGkFjYax3UXc9BekCmx2CdvyfDfM00EnJKuavHPdmLjmkn2qd2k1cCH7Lu08Bs70EIO1637yB42XWgDEm-8ZmzZU/s1600/IMG_5768_v1.JPG" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSJ7KZiowmk7dTB2HYog23ED7ZD43B6-9jeraexWXn_DuFUX8BddqmGkFjYax3UXc9BekCmx2CdvyfDfM00EnJKuavHPdmLjmkn2qd2k1cCH7Lu08Bs70EIO1637yB42XWgDEm-8ZmzZU/s320/IMG_5768_v1.JPG" /></a><br />
<br />
Notice that I have two options: I either agree to let Google track me, or I cancel the request. There is no "just show my location" option. <br />
<br />
As a matter of principle, I don't want Google to be tracking me. I'm aware that Google can offer me all sorts of useful services if I just let it know every little detail of my life, but I prefer to do without them. But now it seems that zooming in on my GPS-derived location has been added to the list of features I can't have. There is no technical reason for this; it didn't used to be the case. But Google has decided that as the price for looking at the map of where I am, I now have to tell them where I am all the time.<br />
<br />
I'm aware that of course my cellphone company knows roughly where I am and who I talk to, and my ISP knows which websites I visit and can see my email (although unlike GMail I don't think they derive any information about me from the contents), and of course Google knows what I search for. But I can at least keep that information compartmentalised in different companies. I suspect that the power of personal data increases non-linearly with the volume and scope, so having one company know where I am and another company read my email means less loss of privacy than putting both location and email in the same pot.<br />
<br />
Hey, Google, stop being evil!<br />
<br />
<b>Update: 20th April 2015</b><br />
<br />
A few days ago a new update to the Google Maps app got pushed, and its now no longer demanding I let Google track me. In fact the offending dialogue box has now been replaced by one with a "No, and stop pestering me" option, so this is an improvement on what they had before.<br />
<br />
Way to go, Google!Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com3tag:blogger.com,1999:blog-5975524006824862804.post-33991262057569487462014-02-22T06:46:00.000-08:002014-02-22T06:46:58.742-08:00A Review of the joint CNN and BBC production: "The War On Terror"<i>The War on Terror</i> is the latest epic in the long-running <i>World War</i> franchise. The previous serial in the franchise, <i><a href="http://tvtropes.org/pmwiki/pmwiki.php/UsefulNotes/WorldWarII?from=Main.WorldWarII">World War II</a>,</i> was <a href="http://squid314.livejournal.com/275614.html">slammed by the critics</a> for its <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/CardCarryingVillain">cardboard-cutout villains</a>, <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/HopeBringer">unrealistic hero</a> and <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/SuperweaponSurprise">poor plot-lines</a>, although it actually achieved decent ratings. <br />
<br />
The first season of <i>Terror</i> started with a <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/Retcon">retcon</a>. At the end of <i>World War II</i> it looked like the Soviet Union had been set up as the <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/GoodRepublicEvilEmpire">Evil Empire</a> for yet another World War, but the writers seem to have realised that replaying the same plot a third time wasn't going to wow the audience. So at the start of <i>Terror</i> we get a load of <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/BackStory">back story</a> exposition in which the Soviet Union has <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/TheGreatPoliticsMessUp">collapsed</a> for no readily apparent reason, leaving America running a benevolent economic hegemony over the allies from the previous series and also its former enemies, Germany and Japan. There was also mention of a very <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/BigBadassBattleSequence">one-sided Gulf War</a>, apparently to emphasize that America's economic power was still matched by its military, even though it didn't seem to have anyone left to fight. Then in the second episode a bunch of religious fanatics from nowhere flew hijacked airliners into important buildings. While the premise may have been a bit thin the episode managed a level of grandeur and pathos that the franchise hadn't achieved since the <i>Pearl Harbour</i> episode, with the special effects being used to build drama rather than just having huge fireballs. But after this promising start the rest of the season became increasingly implausible, with a <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/OurPresidentsAreDifferent">buffoonish president</a> launching two pointless wars on countries whose governments turned out to have almost nothing to do with the attack he was trying to revenge. The weak plot and unsympathetic characters make the last few episodes of the season hard to watch.<br />
<br />
However in the second season the series <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/GrowingTheBeard">grew a beard</a>. The writers replaced the old president with a good looking black guy who clearly wanted to do the right things, finally giving the audience someone to root for, and the focus switched sharply from armed conflict to corrupt politics. Instead of huge set-piece battles featuring ever-more <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/MoreDakka">improbable weaponry,</a> the drama now focuses on the political situation within America itself. The battles and weapons are still there of course, but no longer driving the plot. Instead the president is shown as a tragic figure as he tries to stop wars, free prisoners and sort out his country's economic problems, but every time some combination of <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/CorruptCorporateExecutive">corporate executive</a>, <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/MorallyBankruptBanker">greedy banker</a> and/or <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/GeneralRipper">General Ripper</a> will block his reforms, sometimes with an <a href="http://tvtropes.org/pmwiki/pmwiki.php/Series/YesMinister">obstructive bureaucrat</a> thrown in for comic relief. He has his hands on the levers of power, but in contrast with his predecessor in <i>World War II</i> those levers don't seem to be connected to anything any more.<br />
<br />
Although each episode stands on its own as a story, several plot arcs are becoming clearer as season 2 draws to a close. Events seem to presage the Fall of the <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/TheRepublic">Republic</a>, a plot similar to the <i>Star Wars</i> prequel trilogy, but much better done. Whereas Lucas' Old Republic was destroyed by a single corrupt ruler who wanted to become <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/EvilOverlord">The Emperor</a>, the American Republic in <i>Terror</i> is being destroyed by the very things that made it strong in the previous series: its industrial capacity, financial power and military strength. This is most clearly seen in the episode <i>Drone Strike</i>, where the president was asked to authorise an attack by a remote controlled aircraft against a suspected terrorist convoy on the other side of the world. America is one of the few countries with the technology and money to field these <a href="http://en.wikipedia.org/wiki/MQ-9_Reaper">unmanned warplanes</a>, and they have become an important part of American power. Then we saw the president's face as he was told that the supposed convoy had actually been a <a href="http://www.theatlantic.com/international/archive/2014/01/the-wedding-that-a-us-drone-strike-turned-into-a-funeral/282936/">wedding party</a>. At the end of the episode he was reduced to defending his actions at a press conference because the people who had got him into this mess were too powerful to sack.<br />
<br />
At the same time there are stories of individual determination and hope set in contrast against the darker backdrop. The recent episode <i>Watching the Watchers</i> showed a soldier and a bureaucrat in different parts of the <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/GovernmentAgencyOfFiction">secret spy agency</a> (or agencies; America seems to have several) independently deciding to rebel against the system they are a part of, by releasing embarrassing secrets to the public. At the same time the episode revealed a hidden factor in previous plot lines. Fans are now reviewing old episodes, even back into the first season, looking for the throwaway lines and improbable coincidences which only now make sense.<br />
<br />
The vision of the writers of <i>Terror</i> is now becoming clear; the real war on terror is not the one being fought with guns and robot aircraft, it is the one being fought in the shadows against a loose and ever-shifting coalition of rich, powerful individuals who have discovered that a <a href="http://en.wikipedia.org/wiki/Culture_of_fear">terrorised population</a> is willing to give them even more money and power, and therefore want to keep it that way. The president's initiatives aren't being blocked by some <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/TheOmniscientCouncilOfVagueness">grand secret conspiracy</a>, its just that all of these people know how to work together if they want stop something happening. But this actually makes them more dangerous; in a conventional conspiracy story the hero just has to find the conspiracy and unmask them, but that isn't going to happen in <i>Terror</i>. In one chilling scene a club of bankers get together for a <a href="http://nymag.com/daily/intelligencer/2014/02/i-crashed-a-wall-street-secret-society.html">party to laugh at the rest of the country</a> for continuing to pay them huge amounts after they have wrecked the economy that they were supposed to be running. A journalist sneaks in and tells the story, but it doesn't make any difference because throwing a party is not a conspiracy. <br />
<br />
So <i>Terror</i> goes into its third season in much better shape than it was at the end of the first. The writers have escaped from the constraints of set-piece battles between huge armies, and found instead a solid theme of individual heroism in a believable world of ambiguous morality and complex politics. It all makes for powerful drama and compelling viewing.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com0tag:blogger.com,1999:blog-5975524006824862804.post-52004662143936769642013-10-11T13:33:00.001-07:002013-10-11T13:39:18.846-07:00TV Resolution FallaciesEvery so often discussion of the ever-higher resolution of TV screens <a href="http://gizmodo.com/5926295/your-tv-is-a-retina-display">generates</a> <a href="http://reviews.cnet.com/8301-33199_7-57366319-221/why-4k-tvs-are-stupid/">articles</a> <a href="http://forum.blu-ray.com/showthread.php?p=1954716">purporting</a> to prove that you can't see the improvement unless you sit a few feet from the largest available screen. Most of these articles make the same three mistakes:<br />
<br />
<b>Fallacy 1: Normal vision is 20/20 vision</b><br />
<br />
The term "20/20 vision" means only that you can see as well as a "normal" person. In practice this means that it is the lower threshold below which vision is considered to be in need of correction; most people can see better than this, with a few acheiving 20/10 (that is, twice the resolution of 20/20).<br />
<br />
<b>Fallacy 2: Pixel size = Resolution</b><br />
<br />
If a screen has 200 pixels per inch then its resolution, at best, is only 100 lines per inch because otherwise you cannot distinguish between one thick line and two separate lines. For the more technically minded, this is the spatial version of the nyquist threshold. Wikipedia has a <a href="http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem">very technical article</a>, but this picture demonstrates the problem:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://upload.wikimedia.org/wikipedia/commons/f/fb/Moire_pattern_of_bricks_small.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://upload.wikimedia.org/wikipedia/commons/f/fb/Moire_pattern_of_bricks_small.jpg" /></a></div>
The pixel pitch is close to the height of a brick, leading to the moire pattern because in some areas the pixels are focused on the middle of a brick and in some areas the pixels are focused on the white mortar.<br />
<br />
So the resolution of the screen in the horizontal or vertical directions is half the pixel pitch. But it gets worse as soon as you have some other angle because those pixels are arranged in a grid. The diagonal neighbours of a pixel are 1.4 times further apart than the horizontal and vertical ones, so the worst-case resolution is the pixel pitch divided by 2*1.4 = 2.8. Call it 3 for round numbers.<br />
<br />
So the conclusion is that the actual resolution of the picture on your screen is about 1/3 of the pixel pitch.<br />
<br />
<b>Fallacy 3: Resolution beyond visual acuity is a waste</b><br />
<br />
The argument here seems to be that if HDTV resolution is better than my eyesight then getting HDTV is a complete waste and I would be better off sticking to my normal standard definition TV.<br />
<br />
Clearly this is wrong: as long as my visual resolution outperforms my TV then I will get a better picture by switching to a higher definition format. <br />
<br />
<b>So when does HDTV become worth it?</b><br />
<br />
20/20 vision is generally considered to be a resolution of 1 arc-minute. If we use the naive approach with all three fallacies then one pixel on a 40 inch HDTV screen subtends 1 arc-minute at a distance of 62 inches, so some articles on the subject have claimed that unless you sit closer than that you don't get any benefit <br />
<br />
However on that 40 inch screen a standard definition pixel will be roughly twice the size (depending on which standard and what you do about the 4:3 aspect ratio on the 16:9 screen), so it will subtend 1 arc-minute at around 124 inches (just over 10 feet). So with 20/20 vision you will be able to separate two diagonal lines separated by one pixel at a distance of 30 feet, and with 20/10 vision that goes out to 60 feet. So if you sit less than 30 feet from a 40 inch screen then you will get a visibly better picture with HDTV than standard definition.<br />
<br />
<b>And what about Ultra HD?</b><br />
<br />
With 20/20 vision you can just about distinguish two diagonal lines one pixel apart on a 40 inch HDTV screen from 15 feet away, and 30 feet if you have 20/10 vision. So if you sit closer to the screen than that then you will get a better picture with Ultra HD. And of course Ultra HD sets are often bigger than 40 inches. If you have a 60 inch set then the difference is visible up to 23 feet away with 20/20 vision and 46 feet with 20/10.<br />
<br />
So higher resolutions are not just marketing hype.<br />
<br />
<b>Final point: Compression artifacts</b><br />
<br />
Digital TV signals are compressed to fit into the available bandwidth. This shows up in compression artifacts; if there is a lot of movement across the image then you may see it become slightly blocky, and if you freeze the image then you can often see a kind of halo of ripples around sharp edges. Higher definition pictures are encoded with more data so that these artifacts are reduced. So even without the increased resolution you may still see an improved picture in a higher resolution format.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com3tag:blogger.com,1999:blog-5975524006824862804.post-69165883784070816592013-05-24T08:08:00.001-07:002013-05-24T08:08:27.165-07:00Elevator pitch for Haskell short enough for an elevator rideGreg Hale has written an "<a href="https://www.fpcomplete.com/user/imalsogreg/functional-programming-elevator-pitch">elevator pitch</a>" for Haskell. While it is certainly a good piece of advocacy, it is quite long, and therefore not an elevator pitch. The idea of an elevator pitch is something you can deliver in the 30 seconds or so that you find yourself sharing an elevator with a potential investor.<br />
<br />
I've been looking for an effective Haskell elevator pitch for some years now, but the only thing I was able to come up with was just that you can deliver software better, faster and cheaper because you need fewer lines of code. This just sounds like hype.<br />
<br />
However I think I've now got something better. Here it is:<br />
<br />
<blockquote class="tr_bq">
Conventional languages make the programmer construct both a control flow and a data flow for the program. There is no way to check they are consistent, and anytime they are inconsistent you get a bug. In Haskell the programmer just specifies the data flow: the control flow is up to the compiler. That simplifies the program, cutting down the work and completely preventing a big class of errors.</blockquote>
Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com5tag:blogger.com,1999:blog-5975524006824862804.post-44643550057958901412013-04-01T02:28:00.001-07:002013-04-01T02:28:19.285-07:00<i>This post originally appeared as a response to <a href="http://www.forbes.com/sites/timworstall/2013/03/31/links-31-march-is-bitcoin-doing-well-or-is-it-just-a-bubble/">this article in Forbes</a>: </i><br />
<br />
Thanks for this article; its good to see some opinions on this subject
backed up with numbers. I still think you are wrong though.
<br /><br />First, your comparison with the US dollar ignores the effect of
fractional reserve banking, which multiplies the ratio of GDP to
monetary base by a factor of around 5. Taking that into account, US GDP
is only around ten times its monetary base. Still a lot more than
Bitcoin, I conceed.
<br /><br />More importantly, Bitcoin is not a normal new currency. A normal
new currency is launched by a government with a territory, citizens,
tax base and GDP. All of these give those trading the currency some
clues to the fundamental value of each unit. Bitcoin has no territory,
citizens or tax base. It has a GDP, but that is dependent on the amount
it is used, and usage seems to be growing. A better way to think of
Bitcoin (as I argue here:
http://paulspontifications.blogspot.co.uk/2013/01/bitcoin-as-disruptive-technology.html)
is as a disruptive technology; at the moment it is principally of use
to those who are poorly served by the incumbent financial industry, but
as it improves it will increasingly move up-market by taking business
from the incumbents. As it does so the Bitcoin GDP will increase by
multiple orders of magnitude, and so therefore will the value of each
Bitcoin.
<br /><br />A bubble is defined by the "bigger sucker" theory; that the
price will keep going up because there will always be someone willing to
pay even more, because the price will keep going up. Bitcoin
investment, on the other hand, is driven by a rational expectation that
Bitcoin use will increase. If one has a rational expectation that
Bitcoin GDP will support a much higher price in a few years time then
buying it now looks like a sensible investment. It might also collapse
in a pile of bits, but as a speculative investment its certainly worth
taking a position in.
<br /><br />Disclaimer: I own some Bitcoins, and I'll think about selling in a couple of years.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com1tag:blogger.com,1999:blog-5975524006824862804.post-35803641180689530522013-02-16T07:21:00.004-08:002013-02-16T07:21:58.150-08:00On Having an E-book ReaderBack <a href="http://paulspontifications.blogspot.co.uk/2011/04/why-i-am-not-buying-ebook-reader.html">in 2011</a> I wrote about the reasons why I wasn't getting an e-book reader. I had found that books generally cost more in e-book format than in dead-tree format, and I was nervous about the digital restrictions management (DRM) that e-books came with. These concerns were only increased when I read about <a href="http://www.bekkelund.net/2012/10/22/outlawed-by-amazon-drm/">Linn Nygaard</a> who had her Amazon account closed (and all her e-books effectively confiscated) for some unexplained violation of Amazon policy that was probably committed by the previous owner of her second-hand Kindle. The fact that her account was restored after 24 hours of world-wide outrage didn't reassure me; fame is fickle, and relying on it as an ally against a giant corporation would be unwise.<br />
<br />
However as a result of that fiasco a <a href="http://www.the-digital-reader.com/2012/06/15/how-add-kindle-drm-removal-plugin-calibre">number</a> <a href="http://arstechnica.com/gadgets/2012/10/drm-be-damned-how-to-protect-your-amazon-e-books-from-being-deleted/">of</a> <a href="http://www.wired.com/gadgetlab/2011/01/how-to-strip-drm-from-kindle-e-books-and-others/">articles</a> were posted about the removal of DRM from e-books using <a href="http://calibre-ebook.com/">Calibre</a>, which is an open-source program for managing your electronic library on your computer and converting between formats. You have to download and manually install some plugins in addition to the standard distribution, but once they are installed you just add a DRMed book to your Calibre library, and it automatically gets the DRM stripped out.<br />
<br />
In parallel with this, a long-running legal case between the US Department of Justice and a number of e-book publishers <a href="http://paidcontent.org/2012/04/16/what-does-the-doj-e-book-pricing-lawsuit-mean-for-readers-now/">resulted in a settlement</a> under which prices have dropped considerably, and are now significantly cheaper than the paperback price.<br />
<br />
So that was my two main objections to an ebook reader dealt with: I could now share books with family in a similar way to a paper copy, and I wasn't paying the publisher to leave out the paper. So I asked for a Kindle for my birthday last year.<br />
<br />
I'm very happy with it. I've downloaded some classics from the <a href="http://www.gutenberg.org/">Gutenberg project</a>, and also picked up some more obscure books like the updated edition of "<a href="http://www.amazon.com/Code-Other-Laws-Cyberspace-Version/dp/0465039146/ref=dp_ob_title_bk">Code and Other Laws of Cyberspace</a>" that I have been wanting to read for a long time. Specialist books like this seem to be a lot cheaper in ebook format, presumably because so much of the cost in the dead-tree version is taken up in the overheads of short-run printing and storage. But even <a href="http://www.amazon.co.uk/Anathem-ebook/dp/B0040QE3A8/ref=sr_1_1_bnp_1_kin?ie=UTF8&qid=1361025833&sr=8-1">Anathem</a> was only £2.99 (although it seems to be rather more when I look at it now).<br />
<br />
My wife tried out my Kindle as well, and asked for one for Christmas. When Amazon proved unable to deliver in time I bought one from the local branch of <a href="http://www.waterstones.com/">Waterstones</a> bookshop. This turned out to be a mistake: the Kindles bought from Waterstones have the nice varied "screensaver" pictures replaced with a fixed bit of <a href="http://www.techdirt.com/articles/20121206/16014421285/buy-your-kindle-waterstones-youre-now-locked-into-one-screensaver-waterstones-logo.shtml">Waterstones branding</a> that can't be replaced (I've come across some instructions for replacing that image by logging into the Kindle using TCP over USB, but that particular back door seems to have been closed now).Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com3tag:blogger.com,1999:blog-5975524006824862804.post-42419349490920015942013-01-30T13:24:00.001-08:002013-01-30T13:26:38.201-08:00Bitcoin as a Disruptive TechnologyOne of the most important ways of thinking about the way that technological and commercial forces create change is Clayton Christensen's notion of a <a href="http://http//www.amazon.co.uk/s/?ie=UTF8&keywords=the+innovator%27s+dilema">Disruptive Technology</a>. According to Christensen, new technologies generally start by serving a niche market that is currently under-served by the incumbent technology (and the companies that sell it). The new technology is ignored by the incumbent companies because they can't see how to make money from it; it doesn't fit their big profitable customers and the niche market is too small to be interesting. So the incumbents ignore it. Meanwhile the new technology matures and improves to the point where it can be used by the big customers, and then suddenly the incumbent technology (and the companies that sell it) are obsolete.<br />
<br />
Like any model of a complex situation this doesn't cover every angle, but its interesting to look at <a href="http://bitcoin.org/">Bitcoin</a> from this perspective. Christensen's approach in his book is to look at the problem from the point of view of a manager in an incumbent company (who sees a disruptive technology as a threat) or a start-up company (who sees it as an opportunity). I'm simply going to look at the major market forces, and in particular the niche markets that Bitcoin might serve better than the incumbents, and the possible paths out from those markets.<br />
<br />
<b>The Underworld</b><br />
<br />
An obvious group of people who are served poorly by the incumbent financial system are criminals. Unlike most markets this is a matter of deliberate design. Over the past century governments have systematically regulated all forms of financial transactions in ways that make it difficult to hide ill-gotten gains, or for that matter legitimately-gotten gains that might incur taxes.<b> </b>Banks are legally required to know who their customers are and report any transactions that seem suspicious, or merely large. For people who move millions of dollars around there are methods to avoid such scrutiny, but these are themselves expensive; the necessary accountants and lawyers don't come cheap. Hence there is a significant group of people who have a pressing need to avoid the scrutiny that comes when you move tens of thousands of dollars around the world, but who can't afford the infrastructure used by those who move millions.<br />
<br />
Traditionally these people have used cash, but that is hard to do across national borders because you have to physically move the stuff, making it vulnerable to interception by the authorities or thieves. So Bitcoin, with its ability to slide across national borders without trace or cost, is very attractive. <br />
<br />
<b>The Grey Market</b><br />
<br />
A related group are those who deal in stuff that is legal in some jurisdictions but not in others. Porn and gambling are two major businesses here. <a href="http://www.guardian.co.uk/society/shortcuts/2013/jan/14/mephedrone-benzo-fury-legal-highs">Certain drugs</a> also fit into this category, but you can't move the product over wires, so it is vulnerable to conventional methods of interdiction (although that doesn't <a href="http://en.wikipedia.org/wiki/Silk_Road_%28marketplace%29">stop everyone</a>).<br />
<br />
Governments trying to control porn and gambling have generally followed the money. This tends not to work well with porn because there is too much available for free. But gambling needs to move money to work, and so <a href="http://en.wikipedia.org/wiki/Online_gambling#Legality">authorities in several countries</a> have attacked it at this point. Hence Bitcoin is <a href="http://arstechnica.com/tech-policy/2013/01/bitcoin-based-gambling-to-expand-in-2013/">very attractive in this market</a> as well; punters can easily convert their local currency into Bitcoins, and if they manage to win something then they can convert their winnings back almost as easily.<br />
<br />
<b>The Unbanked</b><br />
<br />
This is a catch-all term for people who have no access to conventional bank accounts, and hence have to deal in cash or barter. <br />
<br />
Financial solutions for these people have traditionally been expensive and piecemeal. Moving money long distance is done by wire transfer, with hefty charges and the need to physically pick it up. Cash is difficult to hide from brigands and corrupt officials. Credit can be available, but only at punitive interest.<br />
<br />
Things have improved; across Africa variations on themes of <a href="http://www.grameenfoundation.org/what-we-do/microfinance-basics">microfinance</a> and <a href="http://en.wikipedia.org/wiki/M-Pesa">mobile phone banking</a> are changing the lives of millions, but they are still vulnerable. Local laws can <a href="http://tharoor.in/articles/the-crisis-of-microfinance/">limit</a> access, and accounts in local currency are vulnerable to <a href="http://www.telegraph.co.uk/news/worldnews/africaandindianocean/zimbabwe/3167379/Zimbabwe-inflation-hits-231-million-per-cent.html">inflation</a>. A stable currency that can be easily hidden and transferred quickly over long distance could meet a real demand, although it still can't provide credit. <a href="http://www.economist.com/news/finance-and-economics/21569744-use-pre-paid-mobile-phone-minutes-currency-airtime-money">Mobile phone credit</a> is already serving this role in some places, so something that is designed for the job should be readily adopted.<br />
<br />
Actually holding Bitcoins requires rather more computer power than the many third-world mobile phones can provide. But that is unlikely to be a problem for long. If <a href="https://play.google.com/store/apps/details?id=com.pesadroid.free&hl=en">MPESA</a> can have an app, then so can <a href="https://play.google.com/store/apps/details?id=de.schildbach.wallet&hl=en">Bitcoin</a>.<br />
<br />
<b>Conclusion</b><br />
<br />
Bitcoin looks like a classic disruptive technology: it has multiple niches in markets that are under-served by conventional money, and the grey market and the unbanked provide a ready path up to higher-value markets in places that are now reasonably well served by cash and credit cards. The black market will also provide market pull for Bitcoin uptake, but if that were the only early market niche then the mere use of Bitcoins would raise strong suspicion of illegal activity. The presence of legitimate, or at least lawful, uses of Bitcoin provides a rationale for those offering to extend conventional services to Bitcoin users and plausible deniability for those whose Bitcoins have in fact come from illegal operations.<br />
<br />
Accordingly we should expect to see Bitcoin become a strong force in finance over the next decade.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com2tag:blogger.com,1999:blog-5975524006824862804.post-19340534963304801692013-01-30T11:30:00.000-08:002013-02-02T01:52:32.195-08:00Software has CivEng EnvyThere is a <a href="http://www.wired.com/opinion/2013/01/code-bugs-programming-why-we-need-specs/">school</a> <a href="http://books.google.co.uk/books/about/The_unified_modeling_language_user_guide.html?id=caVQAAAAMAAJ&redir_esc=y">of</a> <a href="http://dl.acm.org/citation.cfm?id=804131">thought</a> which says that developing software should be like constructing a building. To make a building you have an architect draw blueprints, and these blueprints are then handed over to a builder who constructs what the architect has specified. According to this school of thought the problem with the software industry is that it doesn't create blueprints before it starts building the software. They look with envy at the world of civil engineering, where suspension bridges and tunnels and tall buildings routinely get finished on budget and schedule.<br />
<br />
This is a superficially attractive idea; software is indeed difficult, and it would indeed be a foolish project manager on a building site who directed the builders to start laying the foundations before the plans had been finalised. But on a closer examination it starts to fall apart.<br />
<br />
Suppose that a big software project does indeed need something analogous to a blueprint before starting on the coding. What, exactly, is a blueprint? What purpose does it serve? And where would that fit into the software lifecycle?<br />
<br />
A blueprint for a building is a precise and complete specification of everything that will go into the building. The builder has the problem of assembling what the blueprint shows, but there is no ambiguity and no variation can be permitted. This is because buildings are safety critical infrastructure. The <a href="http://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collapse">Hyatt Regency walkway collapse</a> was a horrible example of what can happen when someone makes a seemingly innocuous change to the plans for a building. So before a building is constructed the plans have to be approved by a structural engineer who certifies that the building is indeed going to stay up, and by an electrical engineer who certifies that it isn't going to electrocute anyone or catch fire due to an electrical fault, and by a bunch of other engineers with less critical specialties, like air conditioning. The details matter, so the blueprints have to specify, literally, every nut and bolt, their dimensions, the metal they are made from and the torque to which they should be tightened (most of these things are standardised rather than being written down individually for every nut and bolt, but they are still part of the specification). Without this the structural engineer cannot tell whether the building is structurally sound. Similarly the electrical engineer must know about every piece of wire and electrical device. So by the time the blueprints are handed to the builder inventiveness and creativity are neither required nor allowed.<br />
<br />
The only artefact in software development that specifies how the software will operate to this degree of precision is the source code. Once the source code has been written, it is to be executed exactly as written: inventiveness and creativity in its execution are neither required nor allowed. But those who promote the idea of "software blueprints" seem to think that something else, something more abstract, can be a blueprint, and that once these blueprints have been drawn the "construction" of the software (that is, turning these blueprints into source code) can proceed in an orderly and planned fashion, putting one line of code after the next in the manner of a bricklayer putting one brick on top of another.<br />
<br />
But when you look at the <a href="http://en.wikipedia.org/wiki/Unified_Modeling_Language">artefacts</a> that these people proffer, it is clear that they are nothing like precise enough to act as a blueprint; they are more like the artists impressions, sketched floor plans and cardboard models that architects produce during the early phases of design. These artifacts can help people understand how the building will be used and fit into its environment, but they are not blueprints.<br />
<br />
(By the way, the old chestnut about unstable software requirements being
like "deciding that a half-constructed house ought to have a basement"
fails for the same reason. The problem with unstable requirements is
real, but the analogy is wrong).<br />
<br />
But if the blueprint for software is the source code, then the
builder for software is the compiler. This should not be a surprise:
when computer scientists encounter a task that does not require
inventiveness and creativity then their first instinct is to automate
it. If so then it is really civil engineering that needs to be <a href="http://www.smartplanet.com/blog/smart-takes/a-giant-3d-printer-builds-a-livable-house/28301">envious of software engineering</a>.<br />
<br />
Software is not like buildings in other ways too:<br />
<ul>
<li>Buildings have very few moving parts and little dynamic behaviour, whereas software is all about dynamic behaviour (and buildings with dynamic behaviour often <a href="http://en.wikipedia.org/wiki/Tacoma_Narrows_Bridge_%281940%29">have</a> <a href="http://en.wikipedia.org/wiki/Millennium_Bridge,_London#Resonance">problems</a>.</li>
<li>Novelty in buildings is rare. I work in a three storey steel frame office block, which is also on an estate of very similar three storey steel frame office blocks. Software, on the other hand, is almost always novel. If software to do a job is already available then it will be reused; I run Fedora Linux; I don't write my own operating system from scratch. </li>
</ul>
So please can we drop this <a href="http://en.wikipedia.org/wiki/False_analogy">half-baked analogy</a> between writing software and civil engineering. Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com3tag:blogger.com,1999:blog-5975524006824862804.post-71776872194186307672013-01-18T09:30:00.000-08:002013-01-19T02:41:41.273-08:00When Haskell is faster than C<a href="http://damienkatz.net/2013/01/the_unreasonable_effectiveness_of_c.html">Conventional wisdom</a> says that no programming language is faster than C, and all higher level languages (such as Haskell) are doomed to be much slower because of their distance from the real machine.<br />
<br />
<span style="font-size: x-small;"><i>TL;DR: Conventional wisdom is wrong. Nothing can beat highly micro-optimised C, but real everyday C code is not like that, and is often several times slower than the micro-optimised version would be. Meanwhile the high level of Haskell means that the compiler has lots of scope for doing micro-optimisation of its own. As a result it is quite common for everyday Haskell code to run faster than everyday C. Not always, of course, but enough to make the speed difference moot unless you actually plan on doing lots of micro-optimisation.</i></span><br />
<br />
This view seems to be borne out by the <a href="http://benchmarksgame.alioth.debian.org/">Computer Language Benchmarks Game</a> (aka the Shootout), which collects implementations of various programs written in different languages and measures their execution speed and memory usage. The winningest programs are always written in highly optimised C. The <a href="http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=ghc&lang2=gcc">Haskell versions</a> run anything from slightly to several times slower than the C versions. Furthermore, if you read the Haskell it is also highly optimised, to the point where it no longer looks like Haskell. Instead it reads like C translated into Haskell. Based on this, you would be justified in concluding that everyday Haskell (that is, Haskell that plays to the strengths of the language in brevity, correctness and composability) must be irredeemably slow.<br />
<br />
But then I read <a href="http://harmful.cat-v.org/software/OO_programming/_pdf/Pitfalls_of_Object_Oriented_Programming_GCAP_09.pdf">this presentation</a>
by Tony Albrecht, in which he talks about how some seemingly innocent everyday
C++ code is actually very inefficient. Two things in particular caught
my eye:<br />
<ul>
<li>A fetch from the main memory when there is a cache miss costs 400 cycles. Pointer indirection in particular tends to fill up cache lines, and hence increase the number of cache misses overall.</li>
<li>A wrong branch prediction costs over 20 cycles. A branch that skips a
simple calculation can actually run slower than the calculation.</li>
</ul>
To put it another way, C is no longer close to the real machine. The real machine has 3 levels of CPU cache (some of them shared between multiple cores), long instruction pipelines, branch prediction, multiple ALUs and FPUs, and hardware data flow analysis done while the program is being executed in order to schedule all this in a way that makes it look like a simple processor executing one instruction at a time. C doesn't expose all that to the programmer, so it seems to me that the only way to write highly optimized C is to have a sophisticated mental model of the processor and its memory architecture, decide what you want this machine to do, and then reverse-engineer the C which is going to make that happen. The result is difficult to understand and hence hard to maintain. Look at the C implementations of the Shootout problems for examples of what I mean.<br />
<br />
But most code isn't written like that. Tony Albrecht is a games programmer, an expert at squeezing cycles out of the rendering loop. Most developers do not live in that world. For them the objective is to produce code that meets the requirements, which includes being fast enough. This is not laziness or waste, but practicality. First design the optimal algorithm, then implement it in idiomatic code. Only if that does not run fast enough should you start detailed profiling and micro-optimisation. Not only is the micro-optimisation process itself expensive, but it makes the code hard to understand for future maintenance.<br />
<br />
So I wondered: the high level of Haskell gives the compiler many more opportunities for micro-optimisation than C. Rather than comparing micro-optimised programs therefore, it seemed sensible to compare everyday programs of the sort that might be produced when readability is more important than raw speed. I wanted to compare programs that solved a problem large enough to have a useful mix of work, but small enough that I could write Haskell and C versions fairly quickly. After poking around the Shootout website I settled on the <a href="http://benchmarksgame.alioth.debian.org/u32/performance.php?test=revcomp">reverse-complement</a> problem.<br />
<br />
A potential issue was that one of my programs might inadvertently use something highly non-optimal, so I decided I would profile the code and remove anything that turned out to be pointlessly slow, but not change the structure of the program or add complexity merely for the sake of speed. With that in mind I wrote Haskell and C versions. I also downloaded the Shootout winner to get some feel for how my programs compared. You can see my code at the bottom of this post.<br />
<br />
The first version of the Haskell took 30 seconds (compared with the Shootout time of about 0.6 seconds). As I had feared, profiling did indeed reveal something pointlessly slow in it. In order to filter out carriage returns from the input I used "<span style="font-family: "Courier New",Courier,monospace;">isLetter</span>", but unlike the C <span style="font-family: "Courier New",Courier,monospace;">char</span> type the Haskell <span style="font-family: "Courier New",Courier,monospace;">Char</span> covers the whole of Unicode, and determining if one of those is a letter is not trivial. I put the filter after the complement operation and compared the result with zero, which in addition to being faster is also the Right Thing if the input contains invalid characters. Once I had this fixed it dropped down to a much more respectable 4.3 seconds. Interestingly, profiling suggests that about half the time is being spent writing out the 60 character lines; merely printing out the result with no line breaks cut execution down to around 2 seconds.<br />
<br />
The C version, meanwhile, took 8.2 seconds. Profiling did not directly reveal the cause, but it seemed to imply that the processor was spending most of its time somewhere other than my code. I strongly suspect that this time is being spent in <span style="font-family: "Courier New",Courier,monospace;">putc(3)</span> and <span style="font-family: "Courier New",Courier,monospace;">getc(3)</span>. The obvious optimisation would use <span style="font-family: "Courier New",Courier,monospace;">fread(3)</span> and <span style="font-family: "Courier New",Courier,monospace;">fwrite(3)</span> to read and write characters in blocks instead of one at a time, but that would require significant changes to the code; extra bookkeeping to deal with the start of the next sequence (signalled by a "<span style="font-family: "Courier New",Courier,monospace;">></span>" character) when it is found half way through a block, and to insert newlines similarly. Unlike the replacement of <span style="font-family: "Courier New",Courier,monospace;">isLetter</span> in Haskell this would require new variables and control structures driven by the cost of the solution rather than a simple switch to a less expensive expression<br />
<br />
It might be argued that I have tilted the playing field against C by not making these changes, and that any halfway competent C programmer would do so when faced with code that runs too slowly. If <span style="font-family: "Courier New",Courier,monospace;">isLetter</span> is pointlessly slow, isn't the same thing true of <span style="font-family: "Courier New",Courier,monospace;">putc(3)</span> and <span style="font-family: "Courier New",Courier,monospace;">getc(3)? </span>But I think there is a clear <span style="font-family: inherit;">difference. Both programs are written in a character-oriented way because the problem is described in terms of characters. I wrote the inner loop of the C to operate on a linked list of blocks because it looked like a faster and simpler choice than copying the whole string into a new buffer twice the size every time it overflowed (on average this algorithm copies each character once or twice; see Knuth for details). I might have considered reading or writing the characters in reverse order rather than doing the in-memory reverse in a separate function, but profiling didn't show that as a significant time sink. Overall, getting decent performance out of the C is going to take about the same amount of work as writing the code in the first place.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">On the other hand the Haskell has decent performance out of the box because the compiler automates a lot of the micro-optimisation that C forces you to do manually. It may not do as good a job as a human with plenty of time might do, but it does it automatically and reasonably well.</span><br />
<br />
<span style="font-family: inherit;">This isn't principally about code size, but for the record the Haskell has 42 SLOC, of which 21 are executable. The C has 115 SLOC, of which 63 are executable. The Shootout C winner has 70 SLOC, of which 46 are executable (counting comma separated statements as one line per statement).</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">So here is the bottom line. If you <b>really</b> need your code to run as fast as possible, and you are planning to spend half your development time doing profiling and optimisation, then go ahead and write it in C because no other language (except possibly Assembler) will give you the level of control you need. But if you aren't planning to do a significant amount of micro-optimisation then you don't need C. Haskell will give you the same performance, or better, and cut your development and maintenance costs by 75 to 90 percent as well.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<span style="font-family: inherit;">Update: here are the compilation and run commands:</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<span style="font-family: "Courier New",Courier,monospace;">ghc -main-is HaskellText -O2 -fllvm -funbox-strict-fields HaskellText.hs</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<br />
<span style="font-family: "Courier New",Courier,monospace;">gcc -O2 naive-c.c -o naive</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<br />
<span style="font-family: "Courier New",Courier,monospace;">time ./HaskellText < big-fasta.txt > /dev/null</span>
<br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<br />
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;">time ./naive < big-fasta.txt > /dev/null</span></span>
<br />
<a name='more'></a><br />
<span style="font-family: inherit;">For what its worth, here is the code for the two programs.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">First the Haskell:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="background-color: white;"><span style="color: #b45f06;"><span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;">module HaskellText (main) where<br /><br />import Control.Applicative<br />import Control.Monad<br />import Data.ByteString.Lazy.Char8 (ByteString)<br />import qualified Data.ByteString.Lazy.Char8 as B<br />import Data.Char<br />import Data.Int<br />import Data.Vector.Unboxed (Vector, (!), (//))<br />import qualified Data.Vector.Unboxed as V<br />import Data.Word<br />import System.IO<br /><br />-- | A block requiring the reverse complement. The first element is the<br />-- header string. The second element is the actual data.<br />data Block = Block ByteString ByteString<br /><br />main :: IO ()<br />main = getBlocks >>= mapM_ (writeBlock . reverseComplement)<br /><br />-- | Write a block to stdio with line breaks.<br />writeBlock :: Block -> IO ()<br />writeBlock (Block name txt) = do<br /> putChar '>'<br /> B.putStrLn name<br /> mapM_ B.putStrLn $ splitEvery 60 txt<br /><br /><br />-- | A utility function that ought to be part of the bytestring library.<br />splitEvery :: Int64 -> ByteString -> [ByteString]<br />splitEvery n t = go t<br /> where<br /> go t = if (not $ B.null t2) then t1 : go t2 else [t1]<br /> where (t1, t2) = B.splitAt n t<br /><br /><br />-- | Read a series of blocks, each of which starts with a '>' character.<br />getBlocks :: IO [Block]<br />getBlocks = map mkBlock . drop 1 . B.split '>' <$> B.getContents<br /> where <br /> mkBlock bs1 = Block h t<br /> where (h, t) = B.break (== '\n') bs1<br /> <br /><br />reverseComplement :: Block -> Block<br />reverseComplement (Block name codes) = <br /> Block name<br /> $ B.filter (/= '\0')<br /> $ B.map (V.unsafeIndex tbl . ord)<br /> $ B.reverse codes<br /> where<br /> tbl = V.replicate 256 '\0' //<br /> map (first ord) (pairs ++ map (first toLower) pairs)<br /> pairs = [('A', 'T'),('C', 'G'),('G', 'C'),('T', 'A'),<br /> ('M', 'K'),('R', 'Y'),('W', 'W'),('S', 'S'),<br /> ('Y', 'R'),('K', 'M'),('V', 'B'),('H', 'D'),<br /> ('D', 'H'),('B', 'V'),('N', 'N')]<br /> first f (x,y) = (f x, y)</span></span></span></span></span><br />
<br />
<br />
Now the C. Sorry about the missing brockets around the #include files; this blog host won't render them, even when I try writing the HTML directly.<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>
<span style="font-family: "Courier New",Courier,monospace;"><span style="color: #38761d;"><span style="font-size: x-small;">#include stdio.h</span></span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="color: #38761d;"><span style="font-size: x-small;">#include <stdlib .h=".h" font="font" size="2"><span style="font-size: x-small;">stdlib.h</span></stdlib><br />#include strings.h<br /><br /><br />#define BLOCKLEN 1024<br />#define TITLELEN 132<br />#define LINELEN 60<br /><br />typedef struct block_struct {<br /> int len;<br /> char text [BLOCKLEN];<br /> struct block_struct *next;<br />} Block;<br /><br />/* Here to get them profiled */<br />int my_getc (FILE *stream) {<br /> return (getc(stream));<br />}<br /><br />int my_putc (int c, FILE *stream) {<br /> return (putc(c,stream));<br />}<br /><br /><br />/* Initialised by calls to setComplement. Holds the complement codes. */<br />static char complement[256];<br /><br />void setComplement(char c1, char c2) {<br /> complement [c1] = c2;<br /> complement [c2] = c1;<br /> complement [c1 | 0x20] = c2;<br /> complement [c2 | 0x20] = c1;<br />}<br /><br /><br /><br />/* Reverse-complement block in-place */<br />void reverseComplementBlock (Block *block) {<br /> char *ptr1;<br /> char *ptr2;<br /> char c;<br /><br /> ptr1 = &(block->text[0]);<br /> ptr2 = &(block->text[block->len - 1]);<br /> while (ptr1 <= ptr2) {<br /> c = complement [*ptr1];<br /> *ptr1 = complement [*ptr2];<br /> *ptr2 = c;<br /> ptr1++;<br /> ptr2--;<br /> }<br />}<br /><br />/*<br /> Read blocks up to either the next ">" or EOF. Create a chain of<br /> blocks in reverse order of reading.<br />*/<br />Block *readBlocks () {<br /> char c;<br /> int i;<br /> Block *result;<br /> Block *old;<br /><br /> old = NULL;<br /> result = NULL;<br /> c = ' ';<br /> while (c != '>' && !feof (stdin)) {<br /> result = malloc (sizeof (Block));<br /> i = 0;<br /> while (i < BLOCKLEN && !feof (stdin)) {<br /> c = (char)my_getc(stdin);<br /> if (c == '>') {<br /> ungetc (c, stdin);<br /> break;<br /> }<br /> if (c >= ' ') { // Drop line breaks.<br /> result->text[i++] = c;<br /> }<br /> }<br /> result->len = i;<br /> result->next = old;<br /> old = result;<br /> }<br /> return result;<br />}<br /><br /><br />int process_sequence () {<br /><br /> char title[TITLELEN];<br /> Block *ptr;<br /> Block *old_ptr;<br /> int i;<br /> int cpos;<br /><br /> if (!feof(stdin)) {<br /> fgets (title, TITLELEN, stdin);<br /> } else {<br /> return 0;<br /> }<br /><br /> if (index (title, '\n') == NULL) {<br /> while (my_getc (stdin) != '\n') {}<br /> }<br /><br /> ptr = readBlocks ();<br /><br /> printf("%s", title);<br /><br /> cpos = 0;<br /> while (ptr != NULL) {<br /> reverseComplementBlock (ptr);<br /> for (i = 0; i < ptr->len; i++) {<br /> if (cpos++ == 60) {<br /> my_putc ('\n',stdout);<br /> cpos = 1;<br /> }<br /> my_putc (ptr->text[i], stdout);<br /> }<br /> old_ptr = ptr;<br /> ptr = ptr->next;<br /> free (old_ptr);<br /> /* Beware; the obvious "for" loop would free the block and then<br /> dereference it to move to the next one. */<br /> }<br /><br /> my_putc ('\n',stdout);<br /> return 1;<br />}<br /><br />void main () {<br /><br /> int i;<br /> char title[TITLELEN];<br /><br /> for (i = 0; i < 256; i++) {<br /> complement [i] = 0;<br /> }<br /> setComplement ('A', 'T');<br /> setComplement ('C', 'G');<br /> setComplement ('M', 'K');<br /> setComplement ('R', 'Y');<br /> setComplement ('W', 'W');<br /> setComplement ('S', 'S');<br /> setComplement ('V', 'B');<br /> setComplement ('H', 'D');<br /> setComplement ('N', 'N');<br /><br /> while (process_sequence ()) {}<br />}</span></span></span>Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com23tag:blogger.com,1999:blog-5975524006824862804.post-39671151636341022392012-08-06T07:24:00.000-07:002012-08-06T07:24:00.353-07:00The Rise of the Me-Burger?Back in February news broke that a<a href="http://www.guardian.co.uk/environment/2012/feb/19/test-tube-burger-meat-eating?intcmp=239"> university researcher</a> had successfully grown bovine muscle tissue in a lab, although a prototype burger made this way would cost an estimated £200,000.<br />
<br />
I was immediately reminded of an <a href="http://en.wikipedia.org/wiki/The_Food_of_the_Gods_%28short_story%29">Arthur C. Clarke short story</a> (spoiler alert) in which synthetic meat is routinely eaten, but its nature is disguised for marketing purposes. The latest product is the wildly popular "Ambrosia Plus", which turns out to be synthesized human meat.<br />
<br />
Could this be about to come true? Given that synthetic meat is going to cost more than animal meat for the foreseeable future, its only niche is going to be as a status symbol for rich people. As Clarke pointed out, up until now human meat has been almost unobtainable, and in most societies the steps required to obtain it have made it taboo. But soon it seems likely that anyone with the necessary money will be able to have a sample without any ethical concerns (although the squick remains).<br />
<br />
But then, why not go one step further? The original sample from which the synthetic human-burger is grown has to come from someone, and the identity of that someone could become a marketing point. One can envisage a particularly egocentric billionaire offering his guests burgers cloned from himself. Or perhaps famous people will find themselves being offered large sums of money for a biopsy. Would you like to nibble on Naomi Campbell? Or perhaps a bite of Usain Bolt?Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com1tag:blogger.com,1999:blog-5975524006824862804.post-63645906121570647322012-08-04T15:16:00.000-07:002012-08-04T15:16:16.984-07:00Managing Risks in Company ITAnother month, another story of corporate IT going spectacularly wrong. First we had the <a href="http://in.reuters.com/article/2012/06/29/britain-banks-competition-idINL6E8HR8ID20120629">RBS debacle</a>, now it seems that <a href="http://dealbook.nytimes.com/2012/08/03/trading-program-ran-amok-with-no-off-switch/">Knight Capital Group</a> have lost hundreds of millions to a rogue algorithm. Similar events <a href="http://www.guardian.co.uk/business/2011/nov/04/hsbc-computer-failure-customers-cash">have</a> <a href="http://www.bbc.co.uk/news/business-18780537">happened</a> <a href="http://www.bbc.co.uk/news/business-17220690">before</a>. And its <a href="http://www.stockholmnews.com/more.aspx?NID=8105">not</a> <a href="http://www.guardian.co.uk/business/2009/may/11/tillscrash-tesco-closures">just</a> <a href="http://www.theage.com.au/it-pro/it-failure-leaves-bureaucrats-in-black-hole-20111017-1lsjg.html">banks</a> that suffer from such events.<br />
<br />
There are lessons to be learned here. Some of them I draw directly from the events linked above. Others are derived from watching large software-intensive organizations from the inside.<br />
<br />
<h2>
Lesson 1: It happens without warning</h2>
Your business is ticking over nicely. You have just approved the annual IT budget, and your CIO has assured you that everything is green. Then at 6am you get a phone call telling you that your annual profit for this year, and maybe your entire company, has just vanished into thin air courtesy of a computer that you own but quite possibly have never heard of. How can this happen?<br />
<br />
The answer is that computers are non-linear; small changes can have huge consequences. Change a plus to a minus somewhere in a program with 100,000 lines of code (which is fairly typical) and if you are lucky you will get no output, and if you are unlucky you will get the wrong output.<br />
<br />
Mistakes like that happen all the time. As a rough rule of thumb, when programmers type code they make a mistake every 10 lines or so. Everything after that is about finding and removing those mistakes. On top of that you have the mistakes that were baked in at the specification stage (assuming that your software even has a written specification; if you are relying on a programmer having an informal chat with the person who wants the program then you are in even worse shape).<br />
<br />
The only way to prevent this stuff happening is to treat the technology as important. <br />
<br />
<h2>
Lesson 2: Do a Risk Assessment</h2>
Do an inventory of every single program used by the company in regular business. If anyone has a <a href="http://www.strategy-at-risk.com/2009/03/03/the-risk-of-spreadsheet-errors/">spreadsheet file</a> that they regularly use, treat that as a separate program. You may have to do some digging to find these things, but that spreadsheet that some bright intern in Operations invented last year to schedule the truck drivers could be the one that paralyses your entire operation next February 29th.<br />
<br />
When you have done this you will have a depressingly long list that fits into roughly three categories, listed here in ascending order of risk:<br />
<br />
<ol>
<li>Commercial Off The Shelf (COTS) software. For the most part you can treat this as low priority; it tends to be reasonably well tested before it leaves the supplier. Not always, but you can be sure that you have bigger problems elsewhere. HOWEVER take a long look at any configuration or other input files supplied to this software; they tend to be less well controlled and hence more prone to error. Consider putting anything like this into category 3.</li>
<li>Ancient dragons. Twenty or thirty years ago your company commissioned a big piece of software which has since become a key part of your operation. Its poorly documented and maintained by a few aging programmers, but at least they do understand it (until you offer them early retirement in a round of cost-cutting).</li>
<li>Ad-hoc bits of stuff. These hang off the side of categories 1 and 2 like <a href="http://en.wikipedia.org/wiki/Remora">remora fish</a> around sharks. Typically they convert between obsolete protocols, massage data formats, generate specialist reports, generate input tables and configuration files, and other odd jobs. All those spreadsheets will fit in here too, as will any particularly complicated configuration files from item 1. This stuff is risky because it was generally written on the cheap; poorly specified, documented and tested. As a result it tends to be fragile, and nobody quite knows how it all works or how to fix it when it goes wrong.</li>
</ol>
I'm going to call all this stuff "software" for simplicity, even though some of it is not normally considered to be a "computer program". From the risk point of view its all the same.<br />
<br />
Start with the stuff in item 3, and then work upwards. For each piece of software, figure out where the output goes and how important it is. Make a short list of the biggest risks based on the initial analysis, and then produce Value At Risk figure for each bit of software if it either fails to work or produces the wrong answer (the Knight Capital Group algorithm, for instance, would have had zero impact if it had simply done nothing, but the RBS account update was a problem precisely because it did nothing). Don't forget to include reputation, regulatory and compensation costs in the analysis.<br />
<br />
<h2>
Lesson 3: Risk cannot be outsourced</h2>
You've probably got some outsource contracts already. If any of your high-risk software is outsourced, either as software maintenance or operation, then compare the penalty clauses in your contract with your VAR. You will probably find at least one order of magnitude difference, if not two or three.<br />
<br />
In general you cannot pay a supplier enough to take on your risk. If you are not careful you will find that your supplier has all the power to control the service while you have all the responsibility for their failures.<br />
<br />
Consider running your own acceptance tests on external software development. Yes, the supplier has already run a bunch of tests, but you still need to check that it works in your context <i>before</i> it goes live. If that means you need a whole duplicate IT set-up for testing then so be it. I haven't seen an analysis of the Knight Capital Group failure, but I'll bet dollars to doughnuts that lack of test infrastructure was an important element.<br />
<br />
Take a look at Section 2.2.4 in Nancy Leveson's book <a href="http://mitpress.mit.edu/books/full_pdfs/Engineering_a_Safer_World.pdf">Engineering A Safer World</a>. On one level, the Bhopal disaster was about engineering and procedural failure; an essential safety step was omitted during a routine procedure. But behind that was a long history of cost reduction and outsourcing; skilled staff were replaced by external contractors and training programs were cut back over several years until a serious accident became inevitable.<br />
<br />
(Aside: the system safety world has faced similar issues, and tends to have better documented case histories because loss of life generally triggers public scrutiny: you could learn a lot from reading about system safety). <br />
<br />
IT has many of the same properties as a chemical plant; it is complicated, it requires ongoing maintenance and operation by skilled personnel, and a small mistake can cause a disaster. Companies manage IT as a cost centre, and cost centres exist to be minimised. Hence there is always pressure to replace expensive staff with cheaper, lower skilled replacements, or to outsource the whole thing to someone who promises to do it cheaper, maybe in another country. Over time an accident becomes more and more likely, and eventually inevitable. <br />
<br />
<h2>
Lesson 4: Pay attention to process</h2>
Process, meaning the steps your people go through to carry out part of the operation, needs to be considered as a component of the overall system, and treated almost as if it were a piece of software. The difference is that any complicated process will, sooner or later, include a mistake. This was probably a significant component in the RBS debacle.<br />
<br />
Complicated manual processes can be automated. This often means creating another bit of remora software, but that is actually preferable to a manual process.<br />
<br />
Systematise your processes; make sure they are written down and followed. Keep them up to date.<br />
<br />
Pay attention to change control and configuration management. Lots of mistakes stem from the wrong version of some file being used. You should know the version of every piece of software you are using, but equally any kind of configuration file should also be under the same kind of control.<br />
<br />
<h2>
Lesson 5: Listen to your engineers</h2>
Neither managers nor engineers can see the whole story; if you listen only to the managers then you will sleepwalk into disaster. If you listen only to engineers then you will wind up commissioning another Concorde or Advanced Gas Cooled Reactor (both of which were created by senior engineers who wouldn't listen to the accountants). The trick is to listen to both.<br />
<br />
The managers will tell you about how to trim costs. The engineers will tell you why that is a bad idea. They will also tell you about the current problems and hazards.<br />
<br />
Do not trust your management reporting chain to tell you this stuff. No manager wants to take a problem to his boss, so the instinctive response to an engineer or lower level manager with a problem is to solve it quickly, or failing that, put a lid on it. An engineer who says "this is fragile" will usually be told "We haven't got time for that now, but we'll fix it when the current urgent project is finished". Of course there is always another urgent project, and so the fix is postponed indefinitely, and no information about the problem percolates upwards.<br />
<br />
A related issue is "<a href="http://en.wikipedia.org/wiki/Technical_debt">technical debt</a>". This is incurred whenever a project makes an expedient short-term decision (usually to meet a project deadline) that has long term costs. Examples include engineering kludges (such as adding one of those remora boxes I talked about earlier) or skimping on documentation. The analogy is with financial loan; you get a short term productivity boost (the loan), but a long term productivity cost (the interest) until you go back and fix the original short-cut (paying off the loan). Imagine the financial chaos if every software project in your company was allowed to borrow money off the books. Then imagine the technical chaos caused by uncontrolled accumulation of technical debt across all your different systems.<br />
<br />
<h2>
Conclusion</h2>
You can stop your corporate IT blowing up in your face, but it takes attention to the details. If you treat IT as a dumb cost centre (like the staff canteen or building maintenance) then you won't just have a slightly shabby IT service, you will have an unstable foundation for your company that could collapse at any moment.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com4tag:blogger.com,1999:blog-5975524006824862804.post-78555446076031606922012-06-24T13:00:00.002-07:002012-06-24T13:00:43.746-07:00A secure bitcoin deviceBitcoins seem to be here to stay. They are being used an increasing amount, in ways both legitimate and illegitimate. But security is a problem for any system where irrevocable and (almost) untraceable transactions can move significant value. The conventional banking system has evolved a system of traceability that lets it wind back fraudulent or simply erroneous transactions, but bitcoins lack such safeguards. Added to this is the fact that any bitcoin wallet system has to be connected to other untrusted computers in order to be useful (and in practice that usually means the Internet). Malware exists that automatically hunts for bitcoin wallets and empties them.<br />
<br />
In short, keeping BTC on your home PC is about as secure as keeping physical cash in a pot on the mantelpiece while having your house redecorated.<br />
<br />
So what would it take to make a bitcoin wallet secure? The answer is, quite a lot.<br />
<br />
<h3>
Threat Analysis</h3>
Step one of security is a threat analysis: what are you protecting, who are the threat, and how well funded are they?<br />
<br />
<b>What?</b> In this case lets assume that we want to protect a bitcoin wallet for common transactions, but the user has conventional bank accounts, pension fund and so forth holding the majority of their non-physical wealth. So the wallet typically only has the equivalent of $100-$200 in it, enough for a week's groceries. Very occasionally it may have enough for a bigger purchase, say $20,000 to buy a car. Lets also assume that bitcoins are in widespread use (suppose Amazon accepted them) and hence pretty much anyone with sense will have done the usual things to protect their wallets. (If that means buying our solution, then our solution is going to be protecting a lot of money, more on this later). This is also not going to protect people who want to keep large amounts of cash outside of a bank: they will need to take stronger measures; the bitcoin equivalent of a safe bolted to a wall rather than a cashbox in a drawer.<br />
<br />
This leaves out a lot of use-cases: Amazon, in particular, are going to need to keep a float worth many thousands of dollars, if not millions. And behind them are going to be financial institutions with substantial holdings. But at that point custom security becomes feasible. This post is about protecting Joe Sixpack's wallet.<br />
<br />
<b>Who?</b> Lets assume that Joe and Jane Sixpack know enough to keep their wallet physically protected, and can trust the people they let into the house, at least to the point of not picking their pockets. That's not always the case of course, but its a good starting assumption. Similarly we are not going to try to prevent them from transferring money to confidence tricksters. So that limits the threat to the digital equivalent of burglary or pick-pocketing; an untrusted outsider gains access to the wallet and steals the coins from it. In this case that would be various forms of digital intruder, either using real-time hacking or malware.<br />
<br />
<b>How well funded?</b> Not all crime is rational, but it can still be a useful starting point to assume that the threat is a hypothetical rational criminal willing to invest resources in the expectation of a return on their investment. In other words we can assume that the resources available to roughly match the rewards on offer.<br />
<br />
The two strategies available to an attacker are to take whatever cash happens to be in the wallet at the time, or to wait until a substantial sum is transferred in and take that. Given the likely time to wait for Joe and Jane to buy a new car (and even assuming that they pay for it using BTC instead of a debit card), its probably better to take the available cash immediately.<br />
<br />
So the most lucrative form of theft would be a "class break" against all wallets of a particular type, followed by a swift emptying of those wallets before countermeasures could be taken. That would be very lucrative indeed. If you could compromise a million wallets with $100 worth of BTC each, you could take $100,000,000. The actual yield would be smaller due to the need to hide, launder and extract value from the cash. But clearly Joe and Jane Sixpack are going to have to be protected against some extremely well-funded adversaries.<br />
<br />
<h3>
<br />A Dedicated Device</h3>
Rootkits that compromise virtual machines are already available and doing the rounds. So trying to wall off the wallet from the rest of a PC is not going to work. A secure bitcoin wallet has to be based on a dedicated platform. For the same reason this platform is going to need its own physical user interface: having it take orders to transfer money from an untrusted PC is as bad as having the wallet on the PC. So we need a device with enough computing power to send and receive BTC, plus a screen and a numeric keyboard for entering PINs and confirming transactions. When you want to transfer BTC to someone your computer sends the amount and ID of the destination wallet to your device, and the device then asks for independent confirmation of the transaction on its screen. As long as the device security is not compromised it is impossible to extract BTC from the device without a human being agreeing to it.<br />
<br />
This implies a small device with a modest processor, a couple of gigabytes of flash, a keypad, a low-resolution LCD screen and a USB port. This is about the same specification as a cheap mobile phone, suggesting that such a device could be mass produced and sold for a few tens of dollars.<br />
<br />
Clearly such a device is going to need a very high degree of internal security, but, given a well defined protocol for transaction requests from outside, this should not be a problem. There will also need to be a secure path for updated software and corresponding upstream security: the digital signature for software updates in particular would be a very tempting target for an attacker.<br />
<br />
<h3>
Backups</h3>
Clearly the wallet device may be damaged or suffer corruption. One solution would simply be to accept the risk, in the same way we accept that money is lost if a physical wallet gets destroyed in a fire. But computers fail rather more often than that, so a backup is probably necessary.<br />
<br />
The problem is that a backup is also an attack avenue: because of the way bitcoin works, if you can get hold of a copy of someone's wallet then you can empty it using any PC. So any backup has to be just as secure, and yet kept reasonably up to date at the same time.<br />
<br />
One option would be to keep the wallet on two independent SD flash cards configured for RAID 1: if one card fails it can be securely destroyed and replaced, and if the device fails then the cards can be moved to a new one. That just leaves data corruption and physical damage as risks. Corruption risk can be minimised by careful design of the software, such as keeping a known-good copy of the wallet as backup during a transaction and running validity checks before committing to the new one. Physical damage is a sufficiently remote possibility to be tolerable in this application.<br />
<h3>
Other Attacks</h3>
The kind of device described so far can be used to verify that money is being sent to a particular wallet, identified by a string of digits. As long as the user knows the destination wallet ID they can be sure that they have sent the right amount to the right person. But this creates a possible attack: malware on the user's PC could systematically replace receiving wallet IDs with those of the attacker. Thus when Joe Sixpack buys a laptop on Ebay he would see payment details specifying a wallet ID and, not realising that this was not the wallet of the vendor, unwittingly send payment to the thief.<br />
<br />
In theory this can be avoided by a digital certificate tying the wallet to a particular person, to be verified and displayed by the wallet device as part of the transaction authorisation. But that requires a wider public key infrastructure that has so far proven expensive and fragile; it might work for large vendors, but not for small ones.<br />
<br />
<h3>
In the Meantime</h3>
For now most of these measures are not necessary. Bitcoin wallets are sufficiently rare that simple measures, such as keeping your wallet on a dedicated virtual machine, are probably sufficient. But if bitcoins become a widespread and popular form of payment then standardised security solutions will become necessary, and any standardised security will be a target for class breaks that will enable many users to be attacked at once.<br />
<br />
If I were to use bitcoins today I would probably put them on a dedicated Raspberry Pi set to make regular encrypted backups to a share on my regular PC, and write down the (very long and random) key somewhere safe. But not everybody is going to be able to set up such a system themselves.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com3tag:blogger.com,1999:blog-5975524006824862804.post-78422083151701595222012-05-15T13:30:00.000-07:002012-05-15T13:30:12.499-07:00Online lectures, early movies, and the Open UniversityThere is a lot of excitement floating around the blogosphere right now about various experiments in remote learning. Some of the big name US universities are putting substantial amounts of material on line: lectures, course material, basically everything except the diploma. <a href="http://gametheorist.blogspot.co.uk/2012/05/what-my-11-year-olds-stanford-course.html">This post</a> describes how an 11 year old boy took a Stanford University course in game theory (written by his proud Dad, but still...)<br />
<br />
One thing the boy said to his father was "I think the concepts are interesting but the presentation is dull.
Couldn’t they have done animations and things to make it better?". The presentation in question was Powerpoint slides with an occasional lecturer's head.<br />
<br />
This reminds me of the early days of movie making. When the movie camera was first invented it seemed obvious how to film a drama: put some actors on a stage to perform a play, put a camera about where the best seat would be, and film the action. Of course this combined the worst features of film (monochrome, no sound, low resolution) with the worst features of theatre (action a long way away, small stage, static scenery). It didn't take movie makers long to realise that taking the actors out of the theatre and putting the camera in amongst them gave better results. The same goes for video lectures.<br />
<br />
The sad thing is, none of this is new. When I was young there was (and still is) a UK institution called the <a href="http://www.open.ac.uk/">Open University</a>. It was created back in 1969 by the Labour government as part of its anti-elitist education-for-all vision. From the beginning it was designed as a mass-market concept: very few students would be at its physical campus; almost all its work would be done by distance learning, with lectures delivered by television, <a href="http://en.wikipedia.org/wiki/Compact_Cassette">audio cassette</a> (link for too young to know what they were) and any other form of high technology communications medium. So for many years you could get up at 6:00 am and watch a couple of hours of half hour lectures on anything from Mathematical Modelling to Sociology 101. In my teenage years I finally got a TV in my bedroom and set up a timeswitch to turn it on at 6:00am. I still remember the theme tune to Sociology 101: "We socialise and we vandalise, We lock the sick awaaayy, Politicians promises, keep changing every daaayy...".<br />
<br />
The presentation was similar to a normal documentary, albeit with a drastically reduced budget; a combination of talking heads, pictures of the thing being described (Sociology 101 showed lots of deprived multi-storey housing estates of the kind the Americans call "projects"), and various visual aids. <a href="http://www.youtube.com/watch?v=COsIDP1fY90">Here are some examples from 1989</a> (Youtube).<br />
<br />
One I remember in particular; it was explaining trigonometry. First it showed a circle with a radius line revolving around it, and the right angle triangle that resulted. Then it turned the circle end on, so all you could see was a vertical line with the blob at the end of the radius moving up and down. Then it moved the line to the right, and the blob traced out a sine wave. And the lightbulb went on in my head. I knew about sine and cosine for right-angled triangles, and I'd seen that shape called a "sine wave", but until that point I hadn't understood the connection. Now I understood perfectly.<br />
<br />MIT, Stanford and the rest need to get back to the future (although they can leave out the flares and courderoys).Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com0tag:blogger.com,1999:blog-5975524006824862804.post-13477135681813531122011-11-18T11:45:00.000-08:002011-11-24T13:19:02.122-08:00Enough with the blue LEDs!The first blue LED was created by <a href="http://en.wikipedia.org/wiki/Shuji_Nakamura">Shuji Nakamura</a> in 1993. It was a technical triumph, and Nakamura eventually managed to win a $9M bonus (after suing his employer). Before then LEDs ranged from red to a slightly yellowish green, and nobody could figure out how to get the wavelengths any shorter. Nakamura's LED was not merely slightly blue; it could go right up into the UV, and was amazingly bright. Today I am typing this in a room illuminated by white LEDs that are made using this technology. Slim, low powered LED-illuminated screens have been created, and blu-ray players use lasers with the same basic semiconductor combination to give us high definition movies on a disk. Science and technology march on, hand in hand.<br /><br />But that's not what I want to write about.<br /><br />On Wednesday I plugged in a new cable modem, enabling faster broadband. On the front panel are two LEDs. One is green and shows that the modem has locked on to a signal. The other is blue and flashes. So now I have this bright blue flashing light near me. The coax cable comes into my office near my desk, so I can't move it far. Fortunately I've been able to tuck it into a cubby-hole next to the bookshelf, so its not in my line of sight. Without that I'd probably have had to put it inside a box or something just to hide the LED.<br /><br />I was not too surprised by the appearance of blue LEDs as an alternative to the old red and green back at the turn of the century; they were novel and eye-catching, which is what manufacturers need if they are to sell stuff. But the trouble is that they are too eye catching; the piercing blue light is actually unpleasant to look at directly, and is far more intrusive than the quiet red and green.<br /><br />Some manufacturers seem to have got the idea. I have a couple of USB hubs with blue LEDs, but they don't flash and they are sufficiently buried inside that you don't have the piercing gleam. But this cable modem (its a Netgear SuperHub, by the way) doesn't follow that rule. We also bought a moderately expensive audio system a few months ago, and that also has a piercingly bright blue LED on the front. So when we watch TV we have this shining in our eyes just below. We've got some relief by sticking some white tape over the LED, but we shouldn't have to do that.<br /><br />So could anyone reading this who works for a consumer electronics company please point the design department at this posting. Remember guys; shining bright flashing lights in your customers' eyes may get their attention, but it won't get you repeat business.<br /><br />I <a href="http://paulspontifications.blogspot.com/2008/03/things-id-put-in-room-101.html">wrote about this</a> before, in 2008. I'm disappointed to have the same problem three years later.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com1tag:blogger.com,1999:blog-5975524006824862804.post-47720908328925086222011-05-22T10:03:00.000-07:002011-05-23T12:09:52.268-07:00Windows, Linux, ARM and Intel in a Zero Sum GameMicrosoft is talking about putting Windows 8 on ARM. Intel is trying to play this down, talking about how Linux is already on ARM, and how Intel even supports Linux itself. None of this makes sense unless you understand the position of Microsoft and Intel in the PC market.<br /><br />People often speak of the "Wintel" duopoly, which of course is a misnomer. A duopoly is when two companies share a single market, at which point there is a risk of anti-competitive behaviour. However Microsoft and Intel are not a duopoly, they are two near-monopolies. Microsoft dominates the desktop operating system market, and Intel dominates the desktop CPU market. Although both OS and CPU are necessary components in a desktop computer, this doesn't make their manufacturers a duopoly because even in the best of worlds they don't compete: increased market share for Microsoft is not at the expense of Intel.<br /><br />However this doesn't mean that everything can be all cozy between them, because in a broader sense they <span style="font-weight: bold;">do</span> compete. Microsoft and Intel are players in the desktop PC "value chain" (actually, its a value network, or even more precisely, a value directed acyclic graph, and if you look closely you find it isn't even really acyclic because chip-makers buy PCs, but term "chain" is more often used so that is what I'll stick to). This describes the way that money flows from PC buyers through vendors to manufacturers to parts makers to raw materials.<br /><br />If you are a player in a value chain then its pretty much a zero sum game; every time someone buys a PC their money bubbles back up through the value chain and everyone grabs their bit. Your problem as a player is to get as much of this money as you can, which inevitably means that someone else gets less. Value chains are characterised by dysfunctional relationships between people who need each other but nevertheless hate each others guts.<br /><br />There are two big strategic goals in a value chain. The first is obvious, the second less so:<br /><ol><li>Make your bit a monopoly, so the rest of the value chain has to come to you to produce the product. That way you can charge monopoly prices and hoover up all the money coming through the value chain.<br /></li><li>Make everyone else's bits into generic commodities so that they can only compete by keeping prices low. That way they <span style="font-style: italic;">can't</span> charge monopoly prices, which leaves more money for you. That's the consequence of the zero sum game thing.</li></ol>Once you understand the second goal you understand everything about Microsoft and Intel.<br /><br /><ul><li>Microsoft makes very sure that Windows runs well on as wide a variety of hardware as possible in order to keep PC hardware a commodity business.</li><li>Intel makes sure that Linux runs nicely on Intel processors in order to create a competitor for Microsoft, so Microsoft will have to reduce its prices, thereby leaving more money for Intel. I suspect Intel were also very supportive of Apple's move to Intel hardware for this reason, as well as the more obvious one of having a new channel.<br /></li><li>Microsoft made sure that Windows runs well on AMD. However AMD have never really recovered from Intel's "Intel Inside" marketing coup (where they paid box vendors to make it sound like "Intel Inside" was a big selling point, so lots of customers thought it was). So now Microsoft need to create a new competitor for Intel, and have decided that ARM will fulfil this role nicely. In the past they also did this with the DEC Alpha for the same reason.</li></ul>ARM already plays nicely with Linux, which is quite a feat because ARM isn't a single processor; its an entire family. If ARM have any sense they will make sure that this continues. Microsoft are likely to try to pay ARM off to get it to remove support for Linux. If ARM have any sense they will resist this because it will enable Microsoft to entrench itself as a mobile monopoly, thereby decreasing the amount of money available for ARM to grab for itself.<br /><br />Edit: "zero sum gain" -> "zero sum game". That's what happens when you let your fingers do the thinking.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com4tag:blogger.com,1999:blog-5975524006824862804.post-65392919160594084042011-05-02T05:26:00.000-07:002011-05-03T10:33:17.115-07:00Patent reduction reduxIn <a href="http://paulspontifications.blogspot.com/2011/04/patent-5893120-reduced-to-mathematical.html">my last post</a> I presented a translation of <a href="http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=5,893,120.PN.&OS=PN/5,893,120&RS=PN/5,893,120">a patent</a> into Haskell. This post went up on <a href="http://news.ycombinator.com/item?id=2501639">Hacker News</a> and <a href="http://tech.slashdot.org/story/11/05/01/1746231/Patent-5893120-Reduced-To-Pure-Math">Slashdot</a>, and I got over 25,000 page views plus a lot of comments. There are too many comments to respond to individually, and many of them cover similar ground. So here is a more general response to the main issues raised.<br /><br /><span style="font-weight: bold;">Its trivial</span><br /><br />Well, yes, in the computer science sense it is. We all know that any algorithm can be written in any turing-complete language, and the Lambda Calculus is turing-complete. QED.<br /><br />However when I read court judgements related to patentability this never gets mentioned, not because judges are too stupid to understand the point, but because as far as I can tell no lawyer has ever tried to put it in front of a judge. I'm not sure exactly why this is, but I suspect that it is a combination of the general squishiness of the law (see below) and the fact that you want to base your case on a tried-and-tested legal theory rather than building something completely new. From a lawyer's point of view an obscure mathematical theorem (most lawyers have never heard of Turing) looks like a very shaky foundation for a legal argument.<br /><br />So it seemed to me that if I could convert this abstract theoretical point about turing-complete languages into a practical demonstration that directly addressed a key point of law then it might make a big difference. Sure its a small step for computer science, but it could be a giant leap for legal thinking.<br /><br /><span style="font-weight: bold;">The law is squishier than that<span style="font-weight: bold;"></span></span><br /><br />Eben Moglen has said<br /><blockquote>... the hacker belief that laws are form of code [creates] a particular frame of analysis for legal questions. [...] one thing which lawyers around the world all share is an awareness of the squishiness of law, it is by no means the hard arthropod carapace for internal soft organs that non-lawyers have a tendency to assume it is</blockquote>Moglen is undoubtedly correct, and when I look at legal judgements I often seem to discern a judge trying to construe the facts and law in a way that will deliver what s/he believes to be justice in the wider human sense as well as in the strict legal sense.<br /><br />But judges are still constrained by the law; however much they might want to, they cannot look at evidence conclusively showing fact X plus a law saying "if X then Y", and then deliver a judgement of "not Y". The trick is to provide incontrovertible evidence for X. To extend Moglen's metaphor, the law may not have an arthropod carapace, but inside the squishy flesh it still has a rigid skeleton of of laws that judges cannot bend at will.<br /><br /><span style="font-weight: bold;">The law / lawmakers are too corrupt; they'll just legislate around this.<span style="font-weight: bold;"></span></span><br /><br />I don't believe the application of the law in the US is anything like corrupt enough for this to be an issue. The worst one can say about the application of American law is that effective legal representation can be bankruptingly expensive for the little guy. But that's not a problem here.<br /><br />Lawmakers are an issue; the leadership of America is routinely for sale to any sufficiently large campaign contribution. But the limits on patentability are long-standing and fundamental, and until the <a href="http://en.wikipedia.org/wiki/State_Street_Bank_v._Signature_Financial_Group">State Street decision</a> in 1998 it was generally understood that computer software and other information-processing algorithms were not patentable. For the US legislature to change this basic principle of patent law seems unlikely.<br /><br /><span style="font-weight: bold;">Everything in the patent is prior art anyway.</span><br /><br />Prior art is one of the most misunderstood things in patent law. Each claim in a patent is an entire invention; there may well be things in the invention that are prior art, but unless you can show that everything in the whole claim existed in a single entity previous to the patent priority date then you don't have prior art.<br /><br />So the fact that hash tables and linked lists were well known before this patent is irrelevant. Of course if anyone can show actual prior art that includes everything in one of the claims then I imagine that the Red Hat lawyers would like to hear of it.<br /><br /><span style="font-weight: bold;">But all patents can be reduced to a formula / number, so this argument would abolish all patents.<span style="font-weight: bold;"><br /></span></span><br />This misunderstands patent law.<br /><br /><a href="http://www.law.cornell.edu/uscode/35/usc_sup_01_35_10_II_20_10.html">USC 35 II 10</a> defines a patentable invention as<br /><blockquote>... any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof ...<br /></blockquote>This definition excludes mathematical formulae and physical facts from patentability. Leaving aside the fact that it is also well known, the quadratic formula is not patentable because it is not a useful process, machine etc.<br /><br />The patent covers the physical realisation of the idea, not the idea itself. So publication of the patented material in any form is fine in theory; that is the whole point of the patent system. What you can't do is make and sell the patented invention. (An exception has been carved out here for ready-to-run implementations of software patents on the grounds that they contribute to infringement by the users, which is bad law but not part of this argument).<br /><br /><span style="font-weight: bold;">The 120 patent recites a computer running the algorithms to create a useful improvement in performance, so that makes it patentable.</span><br /><br />In the State Street case the Court decided that a mathematical formula could form part of an invention, as long as the whole thing was a new and useful process, machine etc. This was a radical departure from previous US decisions, but it matches current European law (I don't know which came first) where mathematical formulae and computer programs cannot be patented "as such", but can be patented as part of an invention that achieves a "technical effect" (which seems to mean the same as "new and useful improvement" in the US).<br /><br />However Bilski has now blown all that out of the water. The Court in Bilski specifically declined to endorse anything in State Street, and instead decided that patentability should be decided in line with three other cases. For more details see <a href="http://paulspontifications.blogspot.com/2010/07/bilski-says-software-is-not-patentable.html">my post on this</a>, but in essence it means that while a mathematical formula or algorithm can form part of a patented invention, it doesn't contribute anything to the novelty of that invention. Even if the formula or algorithm is completely new, for the purposes of the patent it is treated as "well known". Hence in the Diehr case (the rubber curing method) the invention was the use of a calculation to tell the mould when to open rather than a predetermined time interval. The formula to do this was part of the patent, but the invention was its use in a new kind of machine; one that opened based on temperature readings rather than a timer. On the other hand in the Flook and Benson cases the only novelty was in the formula; everything else was held to be already known, so the patents were disallowed.<br /><br />In Bilski the central component of the invention was a formula for hedging bets in energy markets. All else was well known, so the patent was disallowed. The Supreme Court said:<br /><blockquote>Claims 1 and 4 explain the basic concept of hedging and reduce that concept to a mathematical formula. This is an unpatentable abstract idea, just like the algorithms at issue in Benson and Flook.</blockquote>In this case the patent application itself contained the reduction of the concept to a mathematical formula. But clearly if the claims had left the formula unstated and merely listed an equivalent series of steps or assemblage of "means" (e.g. "a multiplication means connected to an addition means") then this would have merely been a change in the drafting, and it is well established that you can't make something patentable just by changing the words in the claims. So this leads to the following conclusion: any element of a patent claim that can be reduced to a mathematical formula is not patentable material, and the subsequent Section 102 analysis (to determine if the invention is new and non-obvious) must ignore it.<br /><br />So in the case of the 120 Patent, all the claims come down to "a general purpose digital computer running software that implements a bunch of mathematical formulae". Running software on a computer is prior art, so there is no new invention here.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com4tag:blogger.com,1999:blog-5975524006824862804.post-42438511042147352042011-04-30T02:17:00.000-07:002011-05-03T23:56:56.602-07:00Patent 5,893,120 reduced to mathematical formulaeI have <a href="http://paulspontifications.blogspot.com/2010/07/bilski-says-software-is-not-patentable.html">previously argued</a> that the Bilski ruling renders all software unpatentable in the US because any method of manipulating information (i.e an algorithm) can be translated into a Haskell program, every Haskell function is a formula in the Typed Lambda Calculus, and all the variants of the Lambda Calculus are part of mathematics. Hence any algorithm can be reduced to a mathematical formula, which US patent law states is not patentable subject matter.<br /><br />The computer science community <a href="http://www.pluto.it/files/meeting1999/atti/no-patents/brevetti/docs/knuth_letter_en.html">has tried</a> to <a href="http://www.groklaw.net/article.php?story=20110426051819346">explain this</a> to the patent industry, but these explanations have fallen on deaf ears. So I thought I would try a different tack.<br /><br />Google has just been <a href="http://www.theinquirer.net/inquirer/news/2045786/google-loses-usd5-million-linux-patent-verdict">ordered to pay $5M</a> for infringing <a href="http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=5,893,120.PN.&OS=PN/5,893,120&RS=PN/5,893,120">patent 5,893,120</a> (hereafter "Patent 120"). This patent covers a very simple data structure and the algorithms for manipulating it. In fact much of the text of the patent is a pseudo-code implementation in a Pascal-like language. So I thought I would provide a practical demonstration of what has, until now, been a theoretical proposition; the reduction of a software patent to set of mathematical formulae. The result is below, and is also <a href="http://hpaste.org/46182/patent__5893120">posted here</a> (although I believe that the paste-bin will eventually expire).<br /><br />I chose this patent partly because it is comparatively simple (there are 95 lines of Haskell source code in the file), but also because it is potentially important; Google's "infringement" consisted of using Linux, which uses this algorithm. Hence anyone using Linux is a potential target.<br /><br />Red Hat is reportedly seeking a declarative judgement that this patent is invalid, and Google is expected to appeal, so this is not over yet. If anyone knows how I can contact their lawyers to draw these formulae to their attention, I'd be grateful.<br /><br /><b>Edit 1:</b>A brief overview of the relationship between Haskell and the Lambda Calculus can be found <a href="http://sucs.org/Knowledge/Articles/Lambda%20Calculus">here</a>.<br /><br /><b>Edit 2:</b>Of course a judge isn't going to know the Lambda Calculus from a lump of rock, but that is what expert witnesses are for. Get a professor of mathematics from an internationally recognised university to testify that these are formulae in the Lambda Calculus, and that the Lambda Calculus is part of mathematics, and you have a sound legal proof. The only thing the patent holders could do is find another professor to testify differently.<br /><br /><b>Edit 3:</b>Sorry about the broken formatting. If you want to review the code you can either <a href="http://hpaste.org/46182/patent__5893120">look at this colourised version</a> or copy and paste it into your favourite editor.<br /><br /><span style="font-weight: bold;">Edit 4: </span>This post has been slashdotted. Rather than try to respond to all the various comments individually I have posted a <a href="http://paulspontifications.blogspot.com/2011/05/patent-reduction-redux.html">general response to the common themes</a>.<br /><br />In the meantime, here are the formulae:<br /><br /><pre><code><br />{- |<br /><br />The claims of United States Patent 5,893,120 may be reduced to a set<br />of mathematical formulae expressed in the Typed Lambda Calculus, also<br />known as System F. This file contains these formulae. The language<br />used is Haskell, which is a version of System F with extra "syntactic<br />sugar" to make it useful for practical purposes.<br /><br />It happens that formulae expressed in Haskell can also be transated by<br />a Haskell compiler into executable computer programs. However that<br />does not make a Haskell function any less of a mathematical formula.<br />Otherwise the quadratic formula<br /><br />> x = (-b +/- sqrt (b^2 - 4ac))/2a<br /><br />would be rendered patentable subject material by the same argument.<br /><br />-}<br /><br /><br />module Patent120 where<br /><br />import Data.Char (ord)<br /><br />-- The "ord" function defines a bijection between text characters and<br />-- numbers.<br /><br />import Data.Map (Map)<br />import qualified Data.Map as M<br /><br />-- The Data.Map module is a standard module defined entirely as a set<br />-- of Haskell formulae, in the same way as this module. The "import"<br />-- means that these formulae are incorporated with the formulae given<br />-- here.<br /><br />{-<br />Claim 1. An information storage and retrieval system, the system<br />comprising:<br /><br /> a linked list to store and provide access to records stored in a<br /> memory of the system, at least some of the records automatically<br /> expiring,<br /><br /> a record search means utilizing a search key to access the linked<br /> list,<br /><br /> the record search means including a means for identifying and removing<br /> at least some of the expired ones of the records from the linked list<br /> when the linked list is accessed, and<br /><br /> means, utilizing the record search means, for accessing the linked<br /> list and, at the same time, removing at least some of the expired ones<br /> of the records in the linked list.<br />-}<br /><br />-- | A record with Birth value below some threshold is considered "expired".<br />type Birth = Integer <br /><br /><br />-- | Records in the list have their birth recorded, and also contain<br />-- a key and a value.<br />data Record k a = Record Birth k a<br /><br />-- | True iff the record is older than the threshold argument.<br />expired t (Record b _ _) = b < t -- | True iff the record matches the key. matches k (Record _ k1 _) = k == k1 -- | The value stored in a record. value (Record _ _ v) = v -- | Records are stored in a linked list in the system memory. type Storage k a = [Record k a] -- | The "search1" function includes a threshold age parameter. -- It returns both the items that match the key and the linked list -- with the expired items removed. -- -- The recitation of "means" cannot be used to disguise the formula -- given here. Otherwise any mathematical formula could be patented -- by reciting e.g. "addition means" and "multiplication means" that are -- mechanically derived from the formula. search1 :: (Eq k) => Birth -> k -> Storage k a -> (Storage k a, [a])<br />search1 t k records = foldr nextItem ([], []) records<br /> where<br /> nextItem r@(Record b k2 v) (retained,found) =<br /> (if b > t then r:retained else retained,<br /> if k == k2 then v:found else found)<br /><br /><br />{-<br />Claim 2. The information storage and retrieval system according to<br />claim 1 further including means for dynamically determining maximum<br />number for the record search means to remove in the accessed linked<br />list of records.<br />-}<br /><br />-- | Similar to "search1", but with an added parameter for the maximum<br />-- number of records to remove.<br />search2 :: (Eq k) => Int -> Birth -> k -> Storage k a -> (Storage k a, [a])<br />search2 n t k = (\(_,x,y) -> (x,y)) . foldr nextItem (n,[],[])<br /> where<br /> nextItem r@(Record b k2 v) (n2,retained,found) =<br /> (n2-1,<br /> if b > t || n2 == 0 then r:retained else retained,<br /> if k == k2 then v:found else found)<br /><br /><br />{-<br />Claim 3. A method for storing and retrieving information records using<br />a linked list to store and provide access to the records, at least<br />some of the records automatically expiring, the method comprising the<br />steps of:<br /><br /> accessing the linked list of records,<br /><br /> identifying at least some of the automatically expired ones of the<br /> records, and<br /><br /> removing at least some of the automatically expired records from the<br /> linked list when the linked list is accessed.<br /><br />Claim 4. The method according to claim 3 further including the step of<br />dynamically determining maximum number of expired ones of the records<br />to remove when the linked list is accessed.<br />-}<br /><br /><br />-- | Claim 3 can be reduced to the same formula as Claim 1. The use of<br />-- a sequence of steps cannot disguise the fact that this represents<br />-- a mathematical formula. Otherwise it would be possible to patent<br />-- any mathematical formula simply by reciting the steps in evaluating<br />-- it. e.g. "step 3: the addition of the results of step 1 and step 2"<br />search3 :: (Eq k) => Birth -> k -> Storage k a -> (Storage k a, [a])<br />search3 = search1<br /><br />-- | Likewise Claim 4 can be reduced to the same formula as Claim 2.<br />search4 :: (Eq k) => Int -> Birth -> k -> Storage k a -> (Storage k a, [a])<br />search4 = search2<br /><br />{-<br />Claim 5. An information storage and retrieval system, the system<br />comprising:<br /><br /> a hashing means to provide access to records stored in a memory of the<br /> system and using an external chaining technique to store the records<br /> with same hash address, at least some of the records automatically<br /> expiring,<br /><br /> a record search means utilizing a search key to access a linked list<br /> of records having the same hash address,<br /><br /> the record search means including means for identifying and removing<br /> at least some expired ones of the records from the linked list of<br /> records when the linked list is accessed, and<br /><br /> meals, utilizing the record search means, for inserting, retrieving,<br /> and deleting records from the system and, at the same time, removing<br /> at least some expired ones of the records in the accessed linked list<br /> of records.<br />-}<br /><br /><br />-- | Every key has a hash code. Strings of characters may be used as<br />-- keys.<br />class (Eq k) => Hashable k where<br /> hash :: k -> Int -- Every key has a hash code.<br /><br />instance Hashable Char where hash = ord<br /><br />instance (Hashable a) => Hashable [a] where<br /> hash = foldr (\x h -> ((hash x + h) * 53 + 1) `mod` 1733) 0<br /><br />type HashedStore k a = Map Int [Record k a]<br /><br /><br />-- | Access a hashed store with a function that returns the modified list of records.<br />hashAccess5 :: (Hashable k) =><br /> (Storage k a -> (Storage k a, [a])) -> Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />hashAccess5 f t k store = (M.insert h retained store, result)<br /> where<br /> h = hash k<br /> (retained, result) = case M.lookup h store of<br /> Just records -> f $ filter (expired t) records<br /> Nothing -> ([], [])<br /><br /><br />-- | Search using hashAccess.<br />search5 :: (Hashable k) => Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />search5 t k = hashAccess5 srch t k<br /> where srch store = (store, map value $ filter (matches k) store)<br /><br />-- | Insert using hashAccess.<br />insert5 :: (Hashable k) => Birth<br /> -- ^ Expiry threshold for old entries.<br /> -> Birth<br /> -- ^ Birth date for new entry.<br /> -> k -> a -> HashedStore k a -> HashedStore k a<br />insert5 t b k v = fst . hashAccess5 (\store -> (Record b k v : store, [])) t k<br /><br /><br />-- | Delete using hashAccess<br />delete5 :: (Hashable k) => Birth -> k -> HashedStore k a -> HashedStore k a<br />delete5 t k = fst . hashAccess5 (\store -> (filter (not . deleted) store, [])) t k<br /> where deleted (Record _ k1 _) = (k == k1)<br /><br /><br />{-<br />Claim 6. The information storage and retrieval system according to<br />claim 5 further including means for dynamically determining maximum<br />number for the record search means to remove in the accessed linked<br />list of records.<br />-}<br /><br /><br />-- | Access a hashed store with a function that returns the modified list of records. The "Int"<br />-- argument is the maxiumum number of expired records to remove.<br />hashAccess6 :: (Hashable k) =><br /> Int -> (Storage k a -> (Storage k a, [a])) -> Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />hashAccess6 n f t k store = (M.insert h retained store, result)<br /> where<br /> h = hash k<br /> (retained, result) = case M.lookup h store of<br /> Just records -> f $ filterN n (expired t) records<br /> Nothing -> ([], [])<br /> filterN _ _ [] = []<br /> filterN n1 p (x:xs) = if n1 <= 0 || p x then x : filterN n1 p xs else filterN (n1-1) p xs -- | Search using hashAccess. search6 :: (Hashable k) => Int -> Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />search6 n t k = hashAccess6 n srch t k<br /> where srch store = (store, map value $ filter (matches k) store)<br /><br />-- | Insert using hashAccess.<br />insert6 :: (Hashable k) => Int<br /> -> Birth<br /> -- ^ Expiry threshold for old entries.<br /> -> Birth<br /> -- ^ Birth date for new entry.<br /> -> k -> a -> HashedStore k a -> HashedStore k a<br />insert6 n t b k v = fst . hashAccess6 n (\store -> (Record b k v : store, [])) t k<br /><br /><br />-- | Delete using hashAccess<br />delete6 :: (Hashable k) => Int -> Birth -> k -> HashedStore k a -> HashedStore k a<br />delete6 n t k = fst . hashAccess6 n (\store -> (filter (not . deleted) store, [])) t k<br /> where deleted (Record _ k1 _) = (k == k1)<br /><br /><br />{-<br />Claim 7. A method for storing and retrieving information records using<br />a hashing technique to provide access to the records and using an<br />external chaining technique to store the records with same hash<br />address, at least some of the records automatically expiring, the<br />method comprising the steps of:<br /><br /> accessing a linked list of records having same hash address,<br /><br /> identifying at least some of the automatically expired ones of the records,<br /><br /> removing at least some of the automatically expired records from the<br /> linked list when the linked list is accessed, and<br /><br /> inserting, retrieving or deleting one of the records from the system<br /> following the step of removing.<br /><br />Claim 8. The method according to claim 7 further including the step of<br />dynamically determining maximum number of expired ones of the records<br />to remove when the linked list is accessed. <br />-}<br /><br />-- | As with Claim 3 vs Claim 1, the formulae for Claim 7 are the same as for Claim 5<br />hashAccess7 :: (Hashable k) =><br /> (Storage k a -> (Storage k a, [a])) -> Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />hashAccess7 = hashAccess5<br /><br />search7 :: (Hashable k) => Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />search7 = search5<br /><br />insert7 :: (Hashable k) => Birth<br /> -- ^ Expiry threshold for old entries.<br /> -> Birth<br /> -- ^ Birth date for new entry.<br /> -> k -> a -> HashedStore k a -> HashedStore k a<br />insert7 = insert5<br /><br />delete7 :: (Hashable k) => Birth -> k -> HashedStore k a -> HashedStore k a<br />delete7 = delete5<br /><br /><br />-- | And the formulae for Claim 8 are the same as for Claim 6<br />hashAccess8 :: (Hashable k) =><br /> Int -> (Storage k a -> (Storage k a, [a])) -> Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />hashAccess8 = hashAccess6<br /><br /><br />search8 :: (Hashable k) => Int -> Birth -> k -> HashedStore k a -> (HashedStore k a, [a])<br />search8 = search6<br /><br />insert8 :: (Hashable k) => Int<br /> -> Birth<br /> -- ^ Expiry threshold for old entries.<br /> -> Birth<br /> -- ^ Birth date for new entry.<br /> -> k -> a -> HashedStore k a -> HashedStore k a<br />insert8 = insert6<br /><br />delete8 :: (Hashable k) => Int -> Birth -> k -> HashedStore k a -> HashedStore k a<br />delete8 = delete6<br /><br /></code></pre>Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com66tag:blogger.com,1999:blog-5975524006824862804.post-78115218557155584892011-04-20T23:52:00.000-07:002011-04-21T00:20:46.447-07:00Why I am voting Yes to Alternative VoteOn May 5th the UK votes whether to change its electoral system from "first past the post" (FPTP) to "alternative vote" (AV). In FPTP you put an X next to the name of one candidate, and the candidate with the biggest number of Xs wins. In AV the first preferences are counted as for FPTP, but then the candidate with the lowest number of votes is eliminated and the second preferences of their voters are then counted towards the other candidates. This process carries on until someone has more than 50% of the vote.<br /><br />When I was at University the Students Union used AV, and one year a man stood for the post of Womens' Officer under the name of "Captain Kirk". Against him were two more conventional candidates standing on the Conservative and Labour platforms.<br /><br />When the votes were counted Captain Kirk got 49% of the first preference votes, with the Conservative getting 26% and Labour 25%. Under FPTP Kirk would have won, but now the Labour candidate was eliminated and it turned out that all of the people who voted for her first had voted for the other woman as second preference. So now the Conservative candidate had 26%+25%=51% of the vote, and Kirk was defeated.<br /><br />I'm well aware of the Arrow Impossibility Theorem, which shows that given an election with three or more candidates and three or more voters it is impossible to have a voting system that always delivers the right result, but it seems to me that this is rather like the fact that any programming language is logically equivalent to a Turing Machine; its true, but it doesn't mean that all (voting systems | programming languages) are equally good. FPTP seems particularly prone to perverse outcomes, such as the election of a male Womens' Officer when the majority of the electorate wants a female. Pretty much every election in the UK is a 2-horse-1-pony race, with the Liberal Democrats as the pony, and every election pundits discuss the impact of "tactical voting" as people who would like to elect the Lib Dems instead vote Labour for fear of otherwise "letting the Conservatives in". Under AV they can simply vote Lib Dem first and Labour second.<br /><br />The anti-AV campaign's arguments seem to come down to:<br /><ul><li><span style="font-weight: bold;">It will create more hung parliaments</span>. Possibly. I don't see this as a bad thing. The current coalition seems to be doing OK, and a coalition means that more of the electorate's views get represented in government. Having a government elected by 40% of the people get 100% of the power seems to me rather undemocratic.</li><li><span style="font-weight: bold;">It will empower the National Front</span>. For those who don't know, the National Front (NF) is an extreme right wing racist party, which occasionally does well in poor inner-city areas where the "immigrants are taking your jobs and houses" line gets a sympathetic hearing. But equally, if most people object to the NF on principle then AV makes it much easier to vote against them; just put the NF at the bottom of your list. That ensures that no matter how other people vote, your vote will count against the NF.</li><li><span style="font-weight: bold;">The most popular person doesn't always win</span>. Well yes, if you define "most popular" as "winner under FPTP" then this is true; if AV didn't give a different result sometimes then there wouldn't be any difference. The point about AV is that the candidates with the broadest support tend to win much more often, whereas FPTP is prone to producing winners who most of the electorate actively dislike.</li></ul>So for these reasons I'm going to be voting YES to AV.Paul Johnsonhttp://www.blogger.com/profile/07353083601285449293noreply@blogger.com5