More on modelling rivers

I've been writing a lot about politics recently - and with reason. But now it's time to be getting back to writing about software, and, specifically, about river flows again.

Computed river map. Ignore the vegetation, it's run only a few generations and does not yet show natural patterns.

I wrote almost a year ago that I had had the first glimmer of success with modelling river flows. Well, some success was right, but not enough success. I didn't have a software framework in which I could model other things I wanted to model in my world, nor one with which I could play flexibly. I also - because I was working with maps of my fictional world, and not the real world - couldn't assess how well my algorithms were working, particularly as I had persistent diagonal artefacts.

So I decided to restart working in Clojure, using a cellular automaton, and with maps of Britain - maps I'm familiar with. I've written about that, too. From the point of view of modelling settlement, it's working well enough to be promising. So now I'm back to river flows, and once again I'm having 'the first glimmerings of success', and once again, frustrations. The picture above shows where I'm up to.

As you can see, I'm still using the 8Km granularity map. I can now actually compute water flows on the 1Km granularity map, but I haven't yet even tried to render them - it's a massive amount of data. But the 8Km map shows both that my algorithm partly works, and that it really does not work anything like well enough.

What works and what doesn't

What works? You can see the River Clyde from about Biggar down to the Falls of Clyde - but there, it disappears. Bits of the mouth of the Clyde are there, but not joined up. The lower Ayr is not only there, but identified as navigable. All of that is promising. I am getting some bits of some rivers showing up, and in the right place.

What doesn't work? Well, 8Km is a horribly wrong scale for doing this sort of thing. Even 1Km granularity is much too coarse, but attempting to map rivers on an 8Km scale... very few of our rivers have enough flow of water to show up. But worse, I haven't implemented 'laking' (as I did in the earlier, Java implementation). When a river arrives at a local low spot, it cannot flow up hill, so it just vanishes. At this coarse a granularity such local low spots are bound to exist. Actually, they will at any granularity - landscapes are fractal. So I need to rewrite my algorithm to deal with local lowspots - fill them up with water until it spills over and follows the rest of the valley down.

Visualisation of the problem. Numbers represent altitude, intensity of blue represents water flow.

This explains why my map of Britain shows so few rivers. The Spey is clearly visible from Aberlour down to Fochabers; the Tweed is a feature in Peebleshire; the Cree and the Esk are visible and recognised as potentially navigable. In England, the Tyne is entirely missing but the Tees is present and marked as navigable; a bit of the Ribble is recognisable as is a bit of the Mersey; tiny bits of the Thames show up around Oxford; and there's a river running north-south, west of London, that might be the Chess or the Misbourne. The Exe and the Tamar are also present. But several of the major rivers of Britain are not present at all, and many of those that are, are, like the Clyde, present only for parts of their course.

The Algorithm

So what is my algorithm? I had hoped, by this time, to publish all my source code so that you can see it. Unfortunately, when I went through the polite step of running this past my present employers, I hit a snag. Although this is - quite obviously - of no relevance to their business and of no interest to them, they claim that it is their secret, proprietary code. Which means that I and my present employers will part; but in the meantime (and possibly afterwards) I cannot publish this code.

I'm damned if I see why I should not publish a description of the algorithm, though.

Firstly, rainfall. I'm not (yet) trying to model the variability of rainfall. I could; last autumn's algorithm recognised that in middle latitudes the wind is primarily a coriolis wind, from the west. It carries damp air from the ocean to the west, and precipitates that rain as it cools - which it does as it rises over higher ground. So west facing slopes recieve more of the potential rainfall than east facing, highlands than low lands, western areas than eastern. We can verify those statements by looking at actual rainfall maps, and they're fairly easy to model. I have not done that yet; my present code just dumps one unit of rain on every cell of the map.

Then, from each cell, I compute the amount of rain which flows through that cell recursively by summing the flows through each of its neighbours which have a greater altitude. This is the simple-minded observation that water flows down hill. And, of course, it does. Obviously, again, the flow value at which I decide a river exists is arbitrary; for this map, I've shown a river where the flow through the cell is thirty units or above - which is to say, where the cell has collected the rainfall from at least thirty upstream cells.

There are two problems with this algorithm as it exists at present. When water in the real world hits a low point then - disregarding the amount lost to evaporation, which in Britain we can do, although in my fictional world I can't entirely - it builds up in a lake until it overflows the lip of the local hollow and continues. I had hoped that river valleys would show up sufficiently strongly that I could get away without laking, but sadly they don't. At the point the Tweed peters out on my map, the altitude of the last cell in which it is visible is 49 (arbitrary units - a monochrome heightmap gives me only 256 gradations of altitude, and sea level is 10). The altitude of the next cell down valley is 50. Given these are average elevations over an 8Km square, it's not in the least impossible that that's correct, but it stops the flow.

Worse still, the last recognised cell of the Ribble is at elevation 28. The next cell down valley - to which the river should flow - is also 28. The Clyde peters out at elevation 35, and the next cell down valley is also 35. It's inevitably the case that long rivers have sections in which they don't fall 5 metres in 8 kilometres, even disregarding the coarse granularity of my elevation data. But again, my present algorithm doesn't cope.

Parallelism and mutable data

There's a second problem, though. This one is not a modelling problem, it's a software framework problem. I'm working, now, in Clojure, and on the whole I find that extremely comfortable; it's a lovely language in which to write exploratory code. And I've been working with a framework which allows me to map a function across a two dimensional array, passing the function two arguments: the whole array, and the cell on which it's to operate. It returns a new copy of that cell, and the mapping operation constructs a new array from all the new cells. This has served me very well during my exploration of modelling environmental change and human settlement, because this process inherently allows great parallelism. On my eight core desktop machine, I can see 5 cores simultaneously maxed out processing a map.

But, the parallelism works because in Clojure data is immutable. In order to process the flow for each cell, I have to process, recursively, the flow in each cell upstream of it. But because the map is immutable, I can't cache those results on the map, so I can only return the one cell I was considering; all those partial results upstream are lost and must be recomputed when those cells are themselves considered. That's obviously wasteful, wasting all, and more, of the time I saved by paralellism. I haven't yet, however, worked out a practical way to pass those partial results round so that they don't need to be recomputed. That's an issue; it's part of the tradeoff of using immutable data, and also, possibly, part of the matter of learning better to use functional languages. But it isn't a blocker. While running this on a one kilometre resolution map is costly in time, it does run.

Laking

'Laking' can't easily be hacked onto my present algorithm. If you allow the algorithm to peak over hilltops as it works upstream, you have a halting problem. What is needed is a second pass which finds low spots where there is a flow of greater-than-river-flow value, and reworks from there.

One possible algorithm which I shall explore would then find the lowest adjoining cell and propagate the flow to that cell and so downstream. That isn't strictly laking, that's a  minimal-uphill algorithm; but it would, I believe, more or less accurately model the river flows.

A slight variation on the minimal-uphill algorithm would be to lower the lowest adjoining cell until it is lower than the cell in which the flow has stopped; that's a slightly more natural solution, with a possible risk of creating gorges where there are no gorges. Again, that's something to explore.

The final workaround is true laking. Let the water level rise - in all cells with the same altitude - until a way out is found. What I worry about laking on my relatively coarse models is that I will construct enormous lakes in unlikely places. It's also likely to turn the whole of East Anglia and Lincolnshire into essentially one lake - but if one interprets these large areas of water on flat land as fen rather than lake, that's more or less right..

Progress is being - very slowly - made.

Mind you, if it takes me two years to work out how to map rivers, this huge game world - which I really know I'll never finish in my lifetime - looks even further out of reach.

Parliamentary Questions

On 19th July 2004, Sir Thomas Dalyell Loch, the Eton and Cambridge educated 11th Baronet of the Binns, voted at Westminster to raise the university tuition fee cap to £3,000 per year - for university students in England, only.

Sir Thomas sat then for the constituency of Linlithgow, in West Lothian, in Scotland; his constituents' education was - then as now - governed by the Scottish Parliament at Holyrood. And thus he answered his own West Lothian Question, which has dogged the British constitutional settlement ever since it was first asked, and which has an added frisson in the aftermath of Scotland's failed velvet revolution.

What is being proposed now by David Cameron, is a parliament - the Westminster parliament - which will continue to debate both bills affecting the whole United Kingdom and also bills affecting England only; but with the quirk that Welsh, Northern Irish and Scots MPs will be unable to vote on the English-only bills. This looks, on the face of it, sensible.

It will not work.

Suppose - just suppose - Ed Miliband wins the next Westminster election, but without a majority of English seats. Who then is the English Secretary of State for Health? For Education? For the Environment? for Transport? for Rural Affairs? Fully half of the current UK cabinet have portfolios which cover only England. If Miliband wins a majority in the UK but not in England, he will find himself on the horns of a dilemma.

Either he appoints Labour members to head the English departments, in which case they will none of them have a majority in the chamber to pass any legislation; or else he appoints Tories to his cabinet, in which case fully half of his cabinet are from the opposition. In either scenario, England is quite ungovernable.

You think that's bad? It gets worse.

Baron Smith of Kelvin, Knight of the Most Ancient and Most Noble Order of the Thistle - no, seriously, if I was trying to make this up I would write something less unbelievable - is the unelected grandee parachuted in by Cameron to oversee an enhancement of democracy in Scotland. He's a Lord, you might have guessed from the title. As such he's a member of the Westminster parliament - like most members of the Westminster parliament, wholly unelected by any constituency.

Up until now, of course, that hasn't mattered. But, is he a Scots Lord or an English one? He was born in Glasgow, and his titular demesne is Kelvin. He even owns a Scottish island - Inchmarnock, off Bute. On this basis, you may say he's Scots. But like many other lads o' pairts, he's spent a great deal of his career furth of Scotland. As such, he's one of very many who have debatable peerages.

Should he be able to vote on English-only bills?

Well, should he?

It will not work, will it?

If only English unelected members should be able to vote on English-only legislation, should Scots unelected members be able to vote on Scots legislation?

Aye, right! The other one has bells on it.

Or should unelected members not vote on any national legislation, but only on union legislation? That too would stick in Scotland's democratic, presbyterian craw. A man is, after all, a man for aa that - even if some prince has made him a beltit knight, a marquis, duke or aa that. We are all Jock Tamson's bairns.

The union parliament cannot be the same institution as the English parliament, for the reasons I've given above. The English parliament must be a separate institution. Whether it's still elected by the unsatisfactory first-past-the-post system, whether it should include an unelected upper house - those are questions for the English. It's not something we Scots should intrude on. Whether the English parliament or the union parliament should inhabit Pugin's gothic extravagance on the bank of the Thames is a decision we can all put off until 2020, since the building is going to be closed for renovation anyway.

But that still leaves two uncomfortable, unanswered questions.

First: are we, in the twenty-first century, still going to tolerate unelected members in the union parliament?

Think about it.

• They're there because some distant ancestor was a ruthless mercenary; or
• They're there because they had once been elected, but the electorate had kicked them out; or
• They're there because they hold office in the English state superstition; or
• They're there because they paid some political party an awful lot of money.

Is that tolerable in today's world?

But second: how do you make a confederation work when one confederate has five times the size and five times the votes of all the others put together?

Of course, this has always been the problem with the United Kingdom, since its inception. Scotland, Wales and Northern Ireland will always - on every vote - be outvoted.  This isn't like the EU or the USA, where smaller states can form coalitions of common interest against the larger ones. England's interests will always prevail. We might just as well not bother to turn up.

Is that tolerable in today's world?

Well, is it?

---

[Some parts of this essay previously appeared in my essay The West Lothian Question, take two]

Jock Tamson's Bairns

So, we lost.

The important thing to remember is that we all lost, every one of us in Scotland. All Jock Tamsons's bairns, those who voted 'no' just as much as we who voted 'yes'. All of our futures are dimmed, all of our hopes of comfort and prosperity diminished, all of our security eroded. And we are luckier than they. Not for us the slow dawning realisation of how dreadfully, how catastrophically they have erred.

When the value of their homes collapses and they are in negative equity, they will know - for they knew that the United Kingdom government was promoting yet another unsustainable housing bubble - they will know they voted for that.

When they cannot afford to pay for their child's cancer treatment, they will know - for they knew that the United Kingdom government was determined to privatise healthcare - they will know that they brought that, too, upon their heads.

When their salaries will no longer pay for adequate food and the shelves at Maryhill are empty, they will know - for they knew the United Kingdom was building a low wage, low skill economy - they will know that that was their choice.

When the terrorist's bomb rips through their train, they will know - for they knew the United Kingdom has made itself, justly, the second most hated nation on Earth - they will know that they chose to step aside from the path of peace.

When there are no jobs in Scotland and their children emigrate in search of work, they will know - for they knew the United Kingdom concentrated all investment within the circle of the M25 - they will know that their action last night made it inevitable.

And when the security state arrests them and holds them without charge, they will know - because they knew the United Kingdom was progressively tearing up human rights - they will know that they have been architects of their own misfortune.

These things will come. The sky to windward is dark.

So we must swallow our anger. Master our grief. Cast away resentment and schadenfreude.

When these things happen - and they will - we must all stand together in solidarity, 'yes' voters with 'no' voters, hope with fear, the brave with the cautious. For we are all - still all, all of us together - Jock Tamson's bairns.

Father, forgive them, for they know not what they do.

A Bill anent Wrongfully Enclosed Common Lands

Right, we've lost the referendum, we won't get independence. It's time to move on. 

As Lesley Riddoch pointed out before this whole campaign began, we can do a great deal to change Scotland for the better with the powers we already have. Up until now, the SNP government have been adopting bland and uncontroversial policies in order not to scare the horses in advance of the vote. I grudgingly accepted that was a reasonable policy - then.

It won't wash now. It's time to make maximal, radical use of our powers, to change Scotland, to redistribute power and wealth from the elite to the people. There are plenty of things we can do, but here is, it seems to me, one particularly low hanging fruit: wrongfully enclosed common lands. And so I present a draft bill. Feel free to improve upon it.

1: Wrongfully Enclosed Common Land

(i) A great deal of land in Scotland which was, historically, held as common, is now held as though it were private property.

(ii) Land is wrongfully enclosed common land if
(a) at some time in the past it was held as common;
(b) rights of common were extinguished without fully and adequately compensating all persons who had rights of common over that land.

(iii) Where it cannot be proven one way or the other whether all persons with rights of common were fully and adequately compensated, it shall be presumed that they were not.

2: Managing agent of Common Lands

(i) Where no other historical arrangements exist for the management of common lands, the managing agent of the common lands shall be
(a) The community council(s) in whose area(s) the lands occur, or
(b) where a community council does not exist, or declares itself unable, for whatever cause, to manage common lands, those lands shall be managed by a common lands factor appointed by the Scottish Government who shall take a management fee of at most [percentage] out of the proceeds and use the remainder for the common good of the people of that area.

(ii) Management of common lands shall be subject to the limitations that
(a) The managing agent may not sell any common land;
(b) The managing agent may not lease out any common land for a term exceeding [period: 20 years?];
(c) The managing agent  may not lease out any common land for a rental less than the fair rental determined by the District Valuer.

(iii) Rental, wayleaves, and other monies levied for use of common land shall be payable to the managing agent and be used by the managing agent for the common good of the people of the area.

3: Voluntary restoration of Common Lands

Where common land has been wrongfully enclosed, but that land has been restored as common land with management passed to the community council(s) in whose area(s) it occurs not later than the date on which this act comes into force, the provisions of sections 4 and 5 of this act shall not apply.

4: Arrangements for smaller holdings

Where greater than 10% of a holding of fewer than 250 hectares in extent is found to be wrongfully enclosed common land,

(i) if no beneficial holder(s) being natural persons can be identified - for example if the land is held by an offshore trust or by a company one or more of whose beneficial shareholders is such a trust - then that part of the holding found to be wrongfully enclosed common land shall be restored as common land, managed as set out in section 2 of this act,
(a) when this act comes into force, or
(b) immediately the land is found to be wrongfully enclosed common land
whichever is the later.
(ii) otherwise, a rent free lease of the wrongfully enclosed common land shall be granted to the natural persons who are beneficial holders
(a) when this act comes into force, or
(b) immediately the land is found to be wrongfully enclosed common land
whichever is the later.
for the period of their lifetimes, after which management shall pass to the managers set out in section 2 of this act.

5: General arrangements

(i) For areas of wrongfully enclosed common lands not covered by sections 3 or 4 of this act, a rental shall be levied at compound interest for the entire period during which such common lands have been wrongfully enclosed.
(ii) The rate of interest shall be 5%.
(iii) The district valuer shall assess a fair rent for the unimproved lands
(a) at the time of the wrongful enclosure and
(b) at twenty-five year intervals thereafter.
(iv) Of this rental
(a) 2.5% shall be payable to the district valuer to cover costs;
(b) 5% shall be payable to the person or agency bringing forward evidence that the lands have been wrongfully enclosed;
(c) The remainder shall be payable to the managing agent for the lands as defined in section 2 of this act.
(iv) This rental shall be payable by the current holder of the lands.
(iv) Management of the lands shall pass to the managing agent for the lands as defined in section 2 of this act
(a) when this act comes into force, or
(b) immediately the land is found to be wrongfully enclosed common land
whichever is the later.

Is a currency union with fUK worth £126 Billion?

Bear with me folks, I'm just thinking aloud. Address it as such.

UK debt is £1.4Tn; the cost of servicing that is £50Bn per year.

Scotland's population share of the debt - if we do the deal that seems to be proposed, of a population share of debt in return for a population share of assets, with the pound Sterling explicitly counted as an asset - is £126Bn. So if we got the same interest rate as the UK gets now, our servicing cost would be £4.5Bn per year, but we'd probably get less good terms so it would be a bit higher - say £5Bn.

Suppose we don't do the deal. Suppose we walk away from the debt. Everyone says we can't do that because the international banks wouldn't lend to us. But, actually, do we need them to?

Scotland's share of the deficit is quoted as £8Bn. Less debt servicing - which we wouldn't have to do because we wouldn't have debt - it's £3Bn. Less the cost of Trident (our share is £2Bn per year), it's £1Bn. Less the cost of overseas military adventures... I don't think Scotland would actually have much of a deficit at all.

So: suppose Scotland actually has no deficit, we don't need to borrow anything from the international banks, so what interest rate they would charge us is purely academic. And suppose in twenty-five years we did need to borrow, by that time we'd have a reasonably good credit rating.

But suppose Scotland does still have a small deficit - say £2Bn a year. Suppose also that the banks charge us four times as much as they currently charge the UK. Then it would take us until 2031 before our deficit hit the £8Bn, which, if we did have an agreement, it would be from day one. And in that time we'd have saved £75Bn.

Suppose we'd made no useful investments, received no interest payments on that saved money, and nothing else had changed, and everything continued as before, it would be 2039 before that £75Bn cushion was exhausted. And if we'd serviced a debt successfully for twenty-three years, we would, inevitably, have earned a reasonable credit rating - perhaps not AAA, but not none. So our interest payments would slacken off.

I'm not saying we should do this. But I honestly don't see that exchanging a currency union for a population share of debt is a good deal for Scotland. Sooner or later we will need a new currency anyway, firstly because Sterling will collapse when the fUK leaves the EU, and secondly because our economy is systematically different from the fUK economy - we are net exporters of food, power and hydrocarbons, they are net importers of all three - so a currency managed for their benefit isn't going to suit us for long. Sooner or later, we need out own currency. And in the short term, no-one denies that Sterlingisation will work.

So I think that we should be robust in our negotiations with fUK, because actually I believe that walking away from the debt is not such a big deal as it's being presented.

Remember: the debt is not Scotland's debt, Scotland is not the nation that contracted it and the nation that did contract it will still exist. If we have clearly and publicly volunteered to take on a fair share of that debt in return for a fair share of assets (including the currency, which is an asset), and that share of assets has been refused, walking away from the debt is not defaulting. The banks may be a bit sniffy about it; we may get our credit rating marked down a bit. But we won't be junk status.