Wednesday, 8 May 2019

Baking the world

Devogilla's Bridge in Dumfries, early foourteenth century
In previous posts, I've described algorithms for dynamically populating and dynamically settling a game world. But at kilometre scale (and I think we need a higher resolution than that - something closer to hectare scale), settling the British Isles using my existing algorithms takes about 24 hours of continuous compute on an eight core, 3GHz machine. You cannot do that every time you launch a new game.

So the game development has to run in four phases: the first three phases happen during development, to create a satisfactory, already populated and settled, initial world for the game to start from. This is particularly necessary if hand-crafted buildings and environments are going to be added to the world; the designers of those buildings and environments have to be able to see the context into which their models must fit.

Phase one: proving - the procedural world

I'm going to call the initial phase of the game run - the phase which takes place before the quest team write their quests and the art department adds their hand-crafted models - 'proving', as when dough has been been made and set aside to rise.

Then, when the landscape has developed - the areas of forest, scrub, open meadow, moorland, savanah and desert are determined, the rivers plotted, the settlers moved in, their trades determined and their settlements allocated, the roadways which link settlements routed, river crossings and ports defined - the proving process ends, and the world is turned over to the plot-writers, quest builders and designers, for a process we can see as analogous to kneading.

But, before going there, to summarise the proving stage. The inputs are:
  1. A raster height map (although this could be randomly generated using any one of many fractal algorithms) - this probably uses ideas from tessellated multi-layer height map;
  2. Optionally, a raster rainfall map at 1km resolution (although my personal preference is that this should be generated procedurally from the height map).
The outputs are
  1. A vector drainage map (rivers);
  2. A raster biome map at roughly 1 km resolution (it might be anything between hectare resolution and 1Km resolution,  but obviously higher resolution takes more storage);
  3. A database of settlers and their settlements, such that the settlements have x,y co-ordinates;
  4. A vector road map.
In this sense, the 'biome map' is just the end state of a Microworld run. The 'biomes' include things like 'forest', 'scrub', 'heath', 'pasture', but they may also include human settlement, and even settlement by different cultural groups.

This gives us all we need to vegetate and furnish the world. When rendering each square metre we have
  1. The x,y coordinates, obviously;
  2. The altitude, taken from the height map;
  3. The biome, taken from the biome map;
  4. The biomes of adjacent cells in the biome map;
  5. The proximity of the nearest watercourse;
  6. The proximity of the nearest road or pathway;
  7. Whether we are inside, or outside, a settlement (where for these purposes, 'settlement' includes enclosed field), and if inside, what type of settlement it is.
Given these parameters, and using the x, y coordinates as seed of a deterministic pseudo-random number generator, we can generate appropriate vegetation and buildings to render a believable world. The reason for pulling adjacent biomes into the renderer is that sharp transitions from one biome to another - especially ones which align to a rectangular grid - rarely exist in nature, and that consequently most transitions from one biome to another should be gradual.

Note that proving, although extremely compute intensive, is not necessarily a one-time job. If the designers aren't satisfied with the first world to emerge from this process, they can run it again, and again, to generate a world with which they are satisfied. It's also possible to hand-edit the output of proving, if needed.

But now, designers and story-writers can see the world in which their creations will be set.

Phase two: kneading - making the world fit our needs

Enough of proving, let's get on to kneading.

Hand-designed buildings and environments are likely to be needed, or at least useful, for plot; also, particularly, very high status buildings are probably better hand designed. I'm inclined to think that less is more here, for two reasons:

You cannot hand design a very large world, it's just impossible. How CD Project Red managed with Witcher 3 I don't know, since I understand that is largely hand designed; but that was a very large team, and even so it isn't a world on the scale I'm envisaging.

Procedurally generated models take a wee bit of compute power to reify, but not a huge amount, and they're trivial to store - you need one single birch leaf model and one single birch-bark texture generator to make every birch tree in the game, and probably a single parameterised tree function can draw every tree of every species (and quite a lot of shrubs and ground-cover plants, too). But once reified, they take no longer to render than a manually crafted model.

By contrast, a manually crafted model will take a very great deal more space to store, such that being able to render a large world from hand crafted models, without excessive model re-use, isn't going to be possible.

So it's better in my opinion to put effort into good procedural generation functions, not just for foliage but also for buildings. My reason for using a picture of a medieval bridge at the head of the essay is to illustrate exactly this point: even in the medieval period, bridges comprise a series of repeating modules. Take one arch module and one ramp module from Devorgilla's bridge as models, add texture skins for several different stone types, stretch the modules a little in whatever dimension is needed, and repeat the arch module as many times as needed, and you can create a range of bridges to span many different rivers - which will all be visibly similar, but that's fine, that's the nature of a traditional culture - but each slightly different.

Take half a dozen sets of models - timber bridges for forested biomes, brick bridges for biomes without stone or timber - and you can build procedural bridges across a whole continent without ever exactly repeating yourself.

However, in some places the designers and story writers will want, for plot reasons and to create iconic environments, to add models. I'm inclined not to over do this, both for reasons of development effort and for reasons of storage cost, but they will. Very high status buildings may need to be unique and distinctive, for example. These need to be designed and their locations and spatial dimensions added to the database, so that the models can be rendered in the right positions (and, critically, procedurally generated models can be omitted in those positions!)

Story and quest writers will also want characters for their plots. While there's no reason why writers cannot add entirely new characters to the database, there's no reason why they cannot incorporate characters generated in the settlement phase into the story; for this reason, characters need to be able to be tagged in the database as plot characters, and with what quests/elements of the plot they're associated.

This allows a mechanism to prevent a plot character from being killed by another non-player character, or dying of disease or starvation, before the plot elements in which they feature have been completed.

Phase three: baking - making it delicious

Once the world has been populated, settled, vegetated, the story has been written, the models built, the quests designed, there is probably a process of optimisation - stripping out things which aren't needed at play time, streamlining things that are - before you have a game ready to ship; but really I haven't yet given that much thought.

Phase four: eating!

At the end, though, you have a game, and a player plays it. How much of the dynamic, organic life that brought the game through proving continues on into the playing phase? If the gossip ideas are to work, if unscripted, non-plot-related events (as well as scripted, plot related events) are to happen while the player plays, if news of these events is to percolate through the world and reach the player in organic, unscripted ways, if a lot of the emergent gameplay I'm imagining is to work, then quite a lot of the dynamic things must be happening.

Of course, part of this depends on the length of 'game world time' is expected to elapse in the course of one play through of the game. If it's less than a year, then you don't need children dynamically being born, and characters dynamically growing older; but if more, then you do. Similarly, you don't need a real simulation of trading to dynamically drive prices in markets, but for a fun trading sub-game to emerge, you probably do, and if you are using merchants as news spreading agents the additional compute cost is not high.

And I understand that many game writers will shudder at the thought that a war might (or might not) start in the middle of their plot, that a battle might, one time in a thousand, take place right where they've plotted some significant encounter. Most modern video games are essentially just very complicated state machines: if you make this sequence of choices, this outcome will happen, guaranteed. Or else they're puddles of random soup, where everything that happens is more or less driven by a random number generator. What I'm envisaging is something quite different: a world in which traders gonna trade, robbers gonna rob, lovers gonna love, scandal-mongers gonna make scandal, organically and dynamically whether the player is there or not, and news of these events will filter through to the player through the gossip network also organically and dynamically.

A world, in short, through which no two runs will ever be the same, in which interesting bits of story will happen with no-one directing or scripting them. And for that to work, some of the same dynamic processes that drove the proving phase have to continue into the eating phase.

Thursday, 11 April 2019

The Changeling

My family in 1960; my father on the right, me next to him
I woke up in the middle of the night thinking about the 1962 Foot And Mouth outbreak, and its effect on my father; wondering to what extent his psychiatric crisis affected mine, and mine, his.

(My father wes then the Ministry of Agriculture's Regional Controller for the North of England; among his responsibilities was deciding which herds were condemned. He insisted on visiting every affected farm, talking to every vet and every farmer. He didn't have to do that. It's because of him that we now know that the disease can be carried on the tyres of cars - specifically our car. He carried the disease from farm to farm with him, infecting farms which had not previously been infected).

To what extent did he undertand the Foot and Mouth outbreak as a punishment for his sin in divorcing his first wife to marry my mother? Or for other sins of his of which I'm unaware? To what extent did both my parents understand my own collapse into a psychiatric basket case - which had started earlier, but got much worse during this period - as a punishment?

Of course, there's no-one else left alive who I can check these thoughts out with, with the exception of my sister who was also only a child at the time. The last of my parents' contemporary friends died last year.

I now know and understand that my father's childhood must have been traumatic, and that the first half of his war - the half he never spoke about - must also have been traumatic. How damaged had he been by these experiences?

As a small child, your own parents are god-like, solid, immutable, the foundations of your existence. It's hard to see them as damaged. Later, in my teens, my psychiatrists saw my own trouble as in part an expression of the conflicts in my parents' marriage, but I now realised that my then understanding of that understanding was shallow.

Both my parents' fathers were in there own way monsters; which is another way of saying both my parents's fathers were in their own way very damaged individuals - but I only dimly understand the causes of their damage. A further generation back, I understand a little of my mother's grandfather's damage, but nothing beyond that. If I had children, they - or their children - would see me as a monster; and they'd be right, of course, in many ways I am one.

Slobodan Milošević was a monster; but his parents marriage failed while he was a small child, and both of his parents committed suicide before he was thirty. I've always seen evil in the world as a consequence of evil, damage resulting from damage, feeding on and reinfecting itself from generation to generation.

This isn't to say my father was a bad man. He wasn't. He was a very brilliant, very troubled man who sincerely strove to be a good man - as I do. But each of his children were badly scarred, as mine would be if I had any, as those to whom I have acted in loco parentis have been.

But so - what? How do we act against evil, if its perpetrators are also its victims?

I have long felt that the appropriate, kind, caring response to children with psychiatric trauma would be to kill them. To put them out of their misery, because you don't recover, because no-one should be forced to live like this. But in saying that I was thinking it simply for the child's sake, to spare the child (and the adult they would grow into) suffering.

What if one should kill children who suffer psychiatric trauma in order to prevent the evil that they may - that they are likely to - cause? Hold onto that thought for a moment. It's a very dark one.

There are folk beliefs about changelings, demon children, cursed children. Does this reflect a traditional practice to driving out or killing very troubled children, and, if so, did that practice have survival benefits for the herd in cauterising evil and limiting its intergenerational spread?

Dark thoughts. I should sleep.

Sunday, 7 April 2019

About taxing the car

My niece and I, in happier times long ago
This is an account of my descent into yet another psyciatric crisis this weekend. It's a bit confusing, mainly because my memory is so scrambled I can't be clear about the order of events; and, of course, it isn't really of interest to anyone else. But writing it down at least lets me get it out. Like talking to a psychiatrist who isn't there, and doesn't answer. Because that's how it is.

Taxing the car

This weekend I had to tax the car. I should have taxed it last weekend, but I hadn't; so it was out of tax, so it was urgent, so I had to do it. The modern system for taxing a car in Britain is very simple and very efficient; the things about bureaucratic websites which usually put me into meltdowns aren't an issue.

What is an issue is I actually can't afford to do this, and in looking at the extent to which I can't afford to I'm forced to face up to just how bad the mess I've spiralled into over the winter has become. My cash income comes from work I do for a wee software company in Scotland. They need me less and less, so the amount I make from it is getting less; now it averages about £500 a month. The software tools I need to do their work cost about £100 a month, and running the office, which I need to do because I don't reliably have enough electricity at home, costs about another £100; so my net income is now around £300 a month. To put it another way, in 1988 I was charging £750 for a day of my time (and getting it); now, ignoring inflation, I'm earning less than that a month.

I like working for these people; they're easy going and don't hassle me, rarely initiate a phone call unless I invite them to, and what they're building isn't in the least evil. Having something regular to do has given me grounding, steadyness, something I have to do for other people. But they don't have enough work for me for this to be viable any longer.

So, I could go out and look for other work. But
  1. I don't have the confidence that I'm any use to anyone. My mental state is so unreliable, and a lot of the time I just can't focus. When I can focus, I'm still quite good, if not as good as I once was. But a lot of the time I can't, and when I get anxious about not being able to deliver work then my focus goes even worse. I can work effectively, on long term average, about fifteen hours a week. But that's long term average, and in the short term I can't guarantee even that.
  2. I really don't cope well with strangers, or, actually, with interacting with people at all - even people I know well and like. Trying to sell myself to people I don't know when I can't honestly claim to be able to deliver for them is something I cannot do.
  3. I cannot claim social security benefits. The degree of stress and humiliation involved is just beyond me. I honestly would rather die.

The phone and the cattle shed

Normally, I refuse to borrow. I know damn fine my sanity is so poor that debts are a strain I just can't deal with. I broke that rule twice last autumn; both times it seemed sensible at the time.

The plan for my croft has been that I keep cattle; to be able to store winter fodder for the cattle, and to be able to feed them easily through the winter, I planned to build a cattle shed. It hadn't made much progress for a couple of years mostly because I lacked the money. Last summer, a friend offered to lend me some money to help getting ahead with it. Initially I refused, but in the autumn when I was really struggling with suicide I thought that if I did borrow some money, I couldn't die until I'd paid it back, and that would act as motivation to keep going. So I borrowed £1,000, and used it to buy concrete blocks and timber. I thought, at the time, I would be able to repay this over a year or so fairly easily.

Then my niece killed herself. She was an orphan, and for complicated reasons she and I were close; we had more of a parent/child relationship than an uncle/niece one. I had to organise the clearing of her flat and her funeral. My phone was broken and I needed a new one urgently to just communicate with people; so I bought one. And between that and the travel and paying for the funeral I went into overdraft.

Which at the time felt justified and also manageable, because my neice left me most of what she had, which isn't a lot but it is something. But I can't get probate on her will until the inquest has been held, and it keeps getting delayed.

I checked my bank account before taxing the car. Usually I pay a year's tax at one go, and that's what I planned to do. But that was impossible; it's obvious to me that there is nothing I can do which will prevent me going over my overdraft limit this month.

Of course, one day the inquest will be held and the death certificate will be issued and I'll be able to get probate on the will and that will, actually, clear my current debts. Except that, it costs money to go on living. I actually can't afford to. Sometime within the next year, even if I get my niece's money, I will run out hard.

Fletcher and the cement mixer

Software is what I do, what I used to be good at, what I use to validate myself. Building software, like any other trade, needs tools, and one of my important tools is Fletcher, my 'big computer', a home built beast with eight processor cores, 16Gb of RAM, and 3Tb of disk, of which 1Tb is SSD. It's not state of the art any more, but it's still pretty fast and provides a really nice working environment; except it also has a relatively high specification video card, and the video card has failed. I think the basic machine, other than the video subsystem, is fine; but I have the security on the mean set up so that you cannot login over the network, which means, without a video subsystem, I cannot log in at all. Which means I cannot really work on the software project I want to work on (even if I had the focus, which I don't).

To do any further work on the cattle shed, I need a cement mixer. Well, I have a cement mixer; but I broke a universal joint on it last autumn. Some day last week I dismantled the drive on the cement mixer to get the universal joint out so that it can be worked on. I think that's when the crash started: I looked at the disassembled parts and thought, OK, that's fine, but I can't take it to an engineer to get it fixed because I can't pay for it.

Project Hope

So, supposing I can go on earning £300 a month, can I survive on that? Well, probably. I spend much more than I need to on telecommunications, I realise it amounts to an addiction. I spend £60 a month on my satellite link, nearly £70 on mobile, and £30 on the landline in the office that hardly works at all. I don't know if I can do the full cold turkey thing and cut myself off from the Internet all together; I've had network comms into my home since before there even was an Internet, back to 300 baud dialup modems, VT100 emulation and JANET addressing. Like software engineering, network comms is part of my identity. The landline in the office ought to go, since it really doesn't work. And, if it goes, for the summer at least the office can go. And that would save a lot.

If I terminate my satellite connection but not my mobile connection, then I'll just end up using more mobile bandwidth, which is really expensive. So it makes more sense, actually, to terminate the mobile - which does of course mean I wouldn't have a phone. But one or other could, and really has to, go.

I spend £30 a month on charity and another about £30 on various Yes movement and news things - Bella, BroadcastScot, Newsnet, the Ferret, Common Weal. Those could go.

But the one that really hurts is Project Hope. Project Hope is the canvassing voter-intention system I built for IndyRef2. It isn't finished and it doesn't work, and I've stopped working on it partly because Fletcher is broken and partly because I no longer believe that independence is going to happen. But it's still sitting there on a server out on one of Linode's web farms which is costing me £40 a month to rent. There's no point in it sitting there. It doesn't work. If Nicola Sturgeon called IndyRef2 tomorrow, it wouldn't be ready and couldn't be finished in time. But switching it off would mean I'd given up, and that all the time and effort I've put into it would be wasted.

Which, actually, it already is, but... knowing it is and facing up to the fact that it is are different.

Facing facts

Intellectually, I know my life is over, just as, intellectually, I know Project Hope has failed. Knowing it and facing it are different.

Any form of suicide takes a certain amount of courage. I used to think about a car crash, but modern cars are much too protective to make that a reliable way out. Two years ago I just stopped taking warfarin, which probably would eventually kill me (but which might just end up with a stroke, which would be even worse). I have various bladed tools with which I could easily kill myself, but I don't think I have the courage for that.

When my lover died last July, I woke up the following morning alone in her house, and spent some time tidying up. In doing so, I took a supply of morphine. I'm not proud of that; it is essentially theft. But I did it, and I have it. It has about a year of shelf-life; if I'm going to use it I'll need to do it soon.

I do not want to die with debts unpaid. But if I continue to live, the debts will accumulate.

I don't really see another alternative.

Friday, 29 March 2019

Brexit: selfishness, short-termism, rent seeking, and hereditary privilege

selfishness, short-termism, rent seeking, and hereditary privilege
Brexit: We got into this mess because some Conservatives couldn't cope with the fact that we weren't the top power in Europe: because, while we won almost all of the votes Europe ever had, it wasn't quite totally all. They couldn't cope with the fact that Germany was more influential than us - because Germany outperforms us at manufacturing, finance, exports, and, critically, diplomacy.

After Brexit, Germany will still outperform us at manufacturing, finance, exports, and, critically, diplomacy - and Germany will be part of the most powerful trading block in the world while we will have NO FRIENDS AT ALL.

The extent of the own goal here is staggering. Brexit will not only not fix any of the things the Brexiteers were upset about, it will make all of them MUCH worse.

The Brexiteers will still blame the same boggarts: they'll blame the EU, they'll blame Germany. But the problems aren't in the EU or Germany, they're in our own disfunctional constitution and our own disfunctional economy. Nobody in Britain wants to face this. We all know it's true, it glares us in the face. Our national governance doesn't work. Our commercial governance doesn't work. Our industrial governance doesn't work. And the reasons are all pretty much the same across all three sectors: selfishness, short-termism, rent seeking, and hereditary privilege.

Our elites don't want to fix these things, because they benefit from them; so they direct the blame outwards. And so the problems don't get fixed. And so the whole British project becomes more creaky, more inefficient, more decrepit, more disfunctional. And we all suffer, including those bloody elites.

Wednesday, 20 March 2019

Kickstarting a hydrogen economy

Battering batteries

There's a lot of interest in battery electric vehicles at present; they're popularly viewed as the future of transport. But there are several problems with the widespread adoption of battery electric vehicles.

  1. The battery techologies required (currently) rely on large quantities of rare minerals, involving substantial environmental and social costs in their extraction;
  2. The batteries themselves are quite heavy, leading to heavier vehicles which have more kinetic energy in a collision and therefore higher propensity for injuries and damage, and, additionally, cause more damage to road surfaces leading to increased need for maintenance (although to be fair battery vehicles are no worse in this regard than conventional SUVs);
  3. Batteries have limited range and take longer to recharge than petrol or diesel vehicles to refuel;
  4. Batteries are inherently quite expensive (although the vehicles have many fewer moving parts and consequently lower manufacturing and maintenance costs than conventional vehicles);
  5. To distribute electrical power to the many charging stations which widespread adoption of battery vehicles would require would need substantial and expensive new grid infrastructure.
But, we urgently need to decarbonise transport (as well as the rest of our economy), so we need to adopt them... but what if there was another technology which would equally decarbonise transport and had none of these problems?

Well, there is.

But before we get to it, let's have a wee digression on Scotland's energy economy, and a glaring inefficiency in it.

Blow wind, and crack your cheeks

Scotland has wind. Scotland has a lot of wind. After only Patagonia, Scotland has the second highest average windspeed in the world. Capturing energy from that glorious surplus of wind should be a no-brainer.

Scotland's electricity grid, which forms part of the UK national grid, was designed essentially as a collection of interconnected hub and spoke networks. Generation was expected to be at the hubs, consumption at the periphery. The revolution in wind power turns that on its head, putting generation at the periphery, and the grid cannot cope. Also, wind blows inconsistently and inconveniently, generating power when it's not needed and not generating power when it is. So in practice, the average Scottish wind turbine spends a lot of its time 'curtailed', which is to say, switched off; chiefly because the radial grid connections which would take its power to consumers are overloaded.

So, in times of strong winds, turbines are often not generating, the energy that they could have generated is being wasted, and the return on the investment which built them is not being returned. But, what if there was a way to embody that energy into something physical which could be conveniently stored and transported?

Well, there is.

The maker of water

Scotland has water. Scotland has a lot of water. After only Norway, Scotland has the second highest average rainfall in Europe.

Take a bucket of water (other containers are available), stick an anode and a cathode into it (these are fancy names for copper rods, although you can use other things), put an electrical potential difference across the anode and cathode, and hydrogen will bubble off at the anode, oxygen at the cathode. Obviously it's desirable to put separate cans over the anode and cathode to collect the gasses separately, since left to themselves they tend to go with a bang, but that's simple, too. If you have surplus electricity and surplus water, you can electrolyse hydrogen virtually for free (you could also make use of the oxygen as a by-product, but I'm assuming for now you simply vent it off to the atmosphere).

Hydrogen can be stored, fairly easily, in fairly inexpensive tanks. Those tanks can be transported. It is flammable, of course, just as petrol is flammable; but industries which have experience of transporting and storing petrol should be able fairly easily to adapt to transporting and storing hydrogen.

This is all simple plant; consequently, it's relatively cheap to make and Scottish engineering industry can easily make it.

Hydrogen passed through a tetraflouroethylene membrane doped with platinum recombines with oxygen from the atmosphere to make pure water, and in the process generates an electric current; the device in which this happens is called a fuel cell. This is, literally, rocket science, but it's not beyond the wit of Scottish manufacturing.

We can dae this.

So, why don't we?

The kicker

There's no market for hydrogen as a fuel, because there are no vehicles on the roads which use it. There's no vehicles on the roads to use it, because you can't conveniently get hydrogen as a fuel. And meantime, most of our wind turbines spend an awful lot of their time doing nothing.

What policy intervention could kickstart this?

A potential answer seems to me very simple. The government could mandate that planning permission would only be given to new windfarms above a given size, provided that the installation included a hydrogen electrolysis plant capable of capturing the entire output of the farm when it would otherwise be curtailed.

Suddenly, windfarm owners would have a lot of hydrogen which they'd want to market; and because there was a lot of it (and it didn't cost much to make - simple plant and otherwise-waste energy) it would be extremely cheap fuel.

The availability of extremely cheap, zero carbon fuel would encourage people and businesses to invest in vehicles which could make use of that fuel (and some government intervention would help here). Such vehicles exist; Honda, Hyundai and Toyota will happily sell you one here in Scotland right now. River simple, from Wales, will lease you one (although at present this is still a prototype scheme).

As adoption of fuel cell vehicles ramped up, the market for hydrogen would expand, motivating owners of existing windfarms to install electrolysis plant.

So what's the downside?

To go back to the beginning:
  1. Batteries require rare minerals. Fuel cells require platinum, which is rare; but not more platinum than is currently used in existing motor vehicle exhausts, and the platinum is easily recyclable. So adoption of fuel cells does not create an additional demand for rare minerals.
  2. Fuel cells are relatively light. Hydrogen storage tanks may be relatively heavy, but not to the extent that it makes the whole vehicle heavier; a fuel cell car should be lighter than an otherwise equivalent battery car.
  3. Hydrogen is a gas, and refuelling should take no longer than LPG refuelling and very little longer than liquid fuel refuelling.
  4. Fuel cells should be very substantially cheaper than batteries.
  5. Hydrogen does not need complex new infrastructure to distribute; on the contrary it can be distributed in tankers just as petrol or LPG are at present.
Obviously, we need to move away from widespread use of private cars. Obviously, we need to move more journeys onto public transport, and to replace much existing physical commuting with tele-commuting. Replacing all existing cars with fuel cell cars would substantially reduce our carbon emissions, but it needs to be part of a much wider revolution.

But in so far as we do still need vehicles, I honestly cannot see a downside to hydrogen.

Nevertheless, this is an area where Scotland is uniquely placed to make a lead.

We can dae this.

Friday, 1 March 2019

We'll not go

Ships leaving the Broomielaw
We’ll not go; we’ll not send back nae letters fae onywhaur -
The ships of migration can ruist on the Clyde.
We’ll not go, we will ficht here, we’ll staun or be buried here,
Lik the floors o the forest on the Broomielaw’s side

Enclosure nae mair,
Eviction nae mair,
Clearance nae mair,
Lairdship nae mair

Dunbar nae mair,
Flodden nae mair,
Pinkie nae mair,
Culloden nae mair.

We’ll not go; we’ll not send back nae letters fae onywhaur -
The time for acceptance has left on the tide
Tak a luik doon the lang road
Fae Caithness tae Gallowa’

Last nicht I lay restless, whiles I wis speirin oan
Whit th Proclaimers hae sang o oor grief an oor pain
For the savour o strawberries that hae grown in thon saut sea
O the tears we hae gret, we hae let faa lik rain.

We’ll not go; we’ll not send back nae letters fae onywhaur -
These times are passed noo, but noo we sall rise
We’ll not go; we sall scrieve here a Claim o Richt for ilkane -
There’s a nacion tae big here, an here we sall bide.

Mundell nae mair,
Carmichael nae mair,
Murray nay mair,
May nae mair.

Province nae mair,
Region nae mair,
Devolution nae mair,
Union nae mair.

Dunbar nae mair,
Flodden nae mair,
Pinkie nae mair,
Culloden nae mair.

Enclosure nae mair,
Eviction nae mair,
Clearance nae mair,
Lairdship nae mair

We’ll not go, we sall bide here, at hame here wi freedom
Whaur the roses an geans noo aa gailie blume
In this yird we will welcome the bairns o auld Adam
Tae aa dance a reel tae a bonnie new tune.

Wednesday, 23 January 2019

Mad software

I was listening to Eric Normand's podcast this morning, as I was making breakfast and tidying my room; he was talking about semantics and data. It started a train of thought which I shall try to unroll.

I have blogged a lot in the past about madness and about software, but I don't think I've ever blogged about madness and software in the same essay. But the reasons I'm mad and the reasons I'm (sometimes) very good at software are related; both have their roots in autism and dyslexia, or, to put it differently, how my brain is wired.

I first wrote about post scarcity software thirteen years ago. It was a thought about how software environments should be designed if were weren't held back by the cruft of the past, by tradition and by a lack, frankly, of anything much in the way of new creative thought. And seeing that the core of the system I described is a Lisp, which is to say it builds on a software architecture which is almost as old as I am, perhaps it is infected by my own take on tradition and my own lack of creativity, but let's, for the purposes of this essay, assume not.

I started actually writing the post scarcity software environment on the second of January 2017, which is to say two years ago. It's been an extremely low priority task, because I don't have enough faith in either my vision or my skill to think that it will ever be of use to anyone. Nevertheless, it does now actually work, in as much as you can write software in it. It's not at all easy yet, and I wouldn't recommend anyone try, but you can check out the master branch from Github, compile it, and it works.

As my mental health has deteriorated, I have been working on it more over the past couple of months, partly because I have lost faith in my ability to deliver the more practical projects I've been working on, and partly because doing something which is genuinely intellectually hard helps subdue the chaos in my mind.

Having said that, it is hard and I am not sharp, and so progress is slow. I started work on big number arithmetic a three weeks ago, and where I'm up to at this point is:
  • addition seems to work up to at least the second bignum boundary;
  • multiplication doesn't work beyond the first bignum boundary;
  • subraction doesn't work, and turns out not to be as easy as just inverting addition;
  • division sort of trivially works, but only in the sense that we can create a rational number out of arbitrary bignums;
  • reading works beyond the first bignum boundary, but not up to the second (because multiplication doesn't work);
  • printing doesn't work beyond the first bignum boundary.
I knew bignums were going to be a challenge, and I could have studied other people's bignum code and have consciously chosen not to do so; but this is not fast progress.

(I should point out that in those three weeks I've also done four days of customer work, which is .Net and boring but it's done, spent two days seeing my sister, spent several days so depressed I didn't actually do anything at all, and done a bit or practical work around the croft. But still!)

In a sense, it wasn't expected to be. Writing the underpinnings of a software environment which is conceptually without limits has challenge after challenge after challenge.

But there are ideas in post scarcity which may have wider utility than this mad idea in itself. Layering homogeneities and regularities onto Clojure maps might - perhaps would - make a useful library, might would make a very useful component for exactly the sort of data wrangling Eric Normand was talking about. Yes, you can use a map - raw data soup - to represent a company. But if this map is a member of a homogeneity, 'Companies', then we know every member of it has employees, and that every employee has a salary and an email address. Regularities and homogeneities form the building blocks of APIs; to use the example Eric discussed in his podcast, the salary is the property of the employee, but the payroll is a property of the company. So in the notation I'm using for post scarcity, you'd get the payroll figure for a company by using a method on the 'Companies' homogeneity. How it computes that value is part of the general doctrine of 'Don't Know, Don't Care': the principal that people writing software at any layer in the system do not need to know, and should not need to care, about how things are implemented in the layers below them.

So, the user needing to find the payroll value for a company would enter something like this:

    (with ((companies . ::shared:pool:companies)
           (acme . companies:acme-widgets-plc))
        (companies:methods:payroll acme))

In practice, in post scarcity notation, the payroll method probably looks something like this:

    (λ (company)
        (reduce + 
            (map ::shared:pool:employees:methods:salary 
                (:employees company))))

There are issues that I haven't resolved yet about the mutability of regularities and homogeneities; obviously, in order to provide multi-user visibility of current values of shared data, some regularities must be mutable. But mutability has potentially very serious perfomance issues for the hypercube architecture, so I think that in general they should not be.

However, that's detail, and not what I'm trying to talk about here.

What I'm trying to talk about here is the fact that if I were confident that these ideas were any good, and that I had the ability to persuade others that they were any good, it would make far more sense to implement them in Clojure and promote them as a library.

But the problem with depression is that you cannot evaluate whether your ideas are any good. The black dog tells me that I'm shit, and that my ideas are shit, and that I don't really know enough to be worth listening to, and that I'm an old tramp who lives in a hut in the woods, and probably smells, and that in any case interaction with other people quickly makes me shaky and confused, and that I can never get my act together, and that I never finish anything.

And all that is objectively true, and I know that it is true. But I also know that I can (or at least have in the past been able to) build really good software, and that I can (or have been able, in the past, to) present ideas really well.

These two collections of statements about me are both true at the same time. But the difference is that I believe the first and I don't believe the second.

And behind all this is the fact that bignum arithmetic is a solved problem. I could dig out the SBCL source code and crib from that. I could read Muhammad ibn Musa al-Khwarizmi, who wrote down the algorithms (the word is a europeanisation of his name) for the general problems of long addition, multiplication and division in around 820AD. I could reread section 4.3 in Knuth.

I am bashing my head against bignum arithmetic and trying to solve it myself, not because it's the most efficient way to produce good code quickly, but because what I'm really trying to do is just distract myself and waste time while I can get on with dying.

And the reason beyond that that I'm working on a software system I know I'll never finish, which is designed to run on computers which don't even exist yet - and although I'm very confident that enormously parallel hardware will be used in future, I'm not at all sure it will look anything like what I'm envisaging - the reason I'm building this mad software is that, because it will never be finished, no-one will ever use it except me, and no-one will say how crap it is and how easily it could have been done better.

Because the other thing that I'm doing in writing this stuff, apart from distracting from the swirling chaos and rage in my head, apart from waiting to die, the other thing I'm doing is trying to give myself a feeling of mastery, of competence, of ability to face problems and solve them. And, to an extent, it works. But I have so little confidence that I actually have that mastery, that competence, that I don't want to expose it to criticism. I don't want my few fragile rags of self worth stripped away.

And so I work, and work, and work at something which is so arcane, so obscure, so damned pointless that no-one will ever use it.

Not because I'm even enjoying it, but just to burn the time.

This is mad.

I am mad.

I hate, hate, hate being mad.

Postscript: just writing this essay has made me tearful, headachey, sick, shaky. It's very hard to face up to the irrationalities and self-deceptions in one's own behaviour.

Wednesday, 2 January 2019

Flodden Field, revisited

Nicola Sturgeon
Scotland has history of meddling in English affairs in the interests of continental friends. It doesn't end well.

In 1513, James IV, quite possibly the best governor Scotland has ever had, tried by diplomacy to persuade England's Henry VIII to make peace with France; and failed. So he sent his navy to France in support of the French, and invaded the north of Northumberland, taking three castles but not much land.

On 4th September, at a council of war with the much larger English army closing and a field of battle agreed, the Earl of Angus - yet another Archibald Douglas - argued that Scotland had done enough for France, it wasn't our fight, and the army should come home. James, annoyed, sent Archie home, with quite a portion of the army. But he then, sensibly, moved the remainder of his army to a very strong position on Flodden Edge, from where the powerful and modern Scottish artillery would have devastating fire over the battlefield.

On the 7th of September the English general, Thomas Howard, Earl of Surrey, wrote to James saying this wisnae fair - in the words of his letter, the Scots position was a fortress. His message was 'come down and fight on the flat, like decent chaps', while carefully preventing the Scots from getting sight of the size of his own army.

So what did James do? He moved his whole army, including those precious guns, to a much weaker position on Branxton Hill; and on Friday the 9th, the English closed. As they did so they brought their own artillery and much of their army in over the Twizell Bridge, within range of the Scottish guns. The Scots did not open fire; chroniclers say that this was because James ordered them not to.

Twizell Bridge, photo By Kirsty Smith, CC BY-SA 2.0
And if you look at the site and it just screams at you. The bridge is still there. It's a single span bridge, extremely vulnerable to heavy artillery, which the Scots army had in position. If the bridge had been destroyed with the English army half across, it would have made a huge difference to the outcome. The Scots would probably have won.

They did not open fire.

Instead, James marched his army down from their position on the hill to meet the English in a marsh; and in that marsh the Scots were butchered.

There were so few Scots survivors from that battle that we do not have good accounts from the Scottish side. James himself, of course, was killed. But we know from the instructions sent to the Scots ambassador in Denmark that the surviving government in Edinburgh saw James' inexperience and folly, rather than any great generalship on the part of the English, as the cause of the disaster, and from what records we have, it's hard to disagree.

But that's not what I came to talk about. I came to talk about the People's Vote.

In March 2017, Nicola Sturgeon, certainly the best governor Scotland has had in modern times, tried by diplomacy to persuade England's Theresa May not to drag Scotland out of the EU against Scotland's will. Like James Stewart before her, she was rebuffed. She had a strong position; Scotland had voted overwhelmingly to remain, she held 56 of the 59 Scottish Westminster seats, she had a clear mandate for an independence referendum.

But Theresa May, like Thomas Howard before her, said it wisnae fair. "Now," she said, "is not the time." And instead she held a general election. The SNP's powerful and modern electoral machine was in a strong position; by standing on an 'Independence Now' manifesto it would have won a majority of Scottish seats and achieved independence while Scotland was still entirely within the EU Aquis Communautaire, making our accession to full membership a formailty.

She did not write that manifesto.

Instead, Nicola marched down off the heights and engaged the Conservatives on their own ground, seeking to keep the entire United Kingdom within the Single Market. As Archibald Douglas said, Scotland had already done our part, by voting remain. England's position in the Single Market or out of it is not our fight.

All Under One Banner, Edinburgh, 6th October.
Of course, she still won a majority of Scottish Westminster seats, but from the position of strength she started from she was bound to; the subsequent eighteen months have shown us how little that majority is worth, without decisive leadership from the top.

Nevertheless, there was still time for an independence referendum before Brexit. Of coourse we couldn't expect a Section 30 letter from Westminster, but that is a technicallity which we could have dealt with in the campaign. The Yes movement was all pepped up and ready to go, the SNP membership was at an all time high.

But no. No, you may not open fire on that exposed bridge, while the English are engaged in complex manouvres. No, we shall meet the English on their own chosen ground, by campaigning on a people's vote.

What is this nonsense? It isn't Scotland's fight! Scotland has voted. It has voted to remain in the EU, and it has voted three solid times in three years to give Nicola Sturgeon the mandate to call a referendum on independence.

Say there's a people's vote. Say Remain wins. How does that help us win independence? It does the opposite. It gives the No campaign the chance to argue once again that the quickest way to lose European citizenship is to vote Yes. So winning this 'people's vote' does not help us.

But that if, as is virtually certain to happen, Scotland again votes Remain, and England again votes Leave? Why, then, we get a reinvigorated right wing government in London ready to repeat the mantra "now is not the time" ad nauseam, and we are at their mercy as they do trade deals with America which will devastate our agriculture, our food and drink industry, our environmental standards and our health service.

Those who will not learn the lessons of history are doomed to repeat them. Just as James Stewart marched down off the hill to the total destruction of his army, his hopes, his nation, over a fight that was never Scotland's to fight, so Nicola Sturgeon is doing once again. The first time is tragedy, agreed, but the second time is too dark to truly be considered farce.

Scotland needs to be better than this. Scotland needs to grow up. Scotland needs to acknowledge that England is different from us, has different needs and priorities to us. Scotland has to learn that we cannot - and should not try to - impose our ways on our bigger neighbour. We have to fight our own battles for our own national interests. And that means, being independent in Europe.

Monday, 3 December 2018

Roads to independence

All Under One Banner - Edinburgh
With the Brexit process nearing its decisive point, I've been reading good analyses of the potential future paths by Chris Grey and Jon Worth - for the UK as a whole. I haven't seen an analysis of the future paths for Scotland which seem as good to me, although the Politics Scotland blog has had a go. I'm not saying my analysis is better or is better informed, but for what it's worth here it is.

No-deal Brexit is probable

Under the European Union (Withdrawal) Act, the UK leaves the EU at 23:00 UTC on 29th March 2019, whether we have agreed deal or not. That is less than four months away now, but it's four months which includes the Christmas and New Year holidays, so in practice it is more like three.

There is no majority in Westminster for the negotiated deal. There is a theory that, having failed in the vote on Wednesday 12th December, May could return to Brussels, negotiate some trivial face-saving tweeks, bring essentially the same document back to parliament and have it pass; I don't believe that is likely.

I think it much more likely that the vote on the 12th will fail, probably heavily; and that in the wrangling and chaos which results, the clock will run out. To me, just now, a no-deal Brexit looks virtually inevitable.

A 'snap election' won't happen

Under the Fixed Term Parliament Act, an early election requires two thirds of MPs to vote for it. Almost half of all MPs are Tories, who know they would be very heavily defeated in an election held now. Turkeys don't vote for Christmas.

A second EU referendum is irrelevant

There's a lot of folk arguing for a second EU referendum. I don't think that will  happen, but it could. For it to happen, all of the following would be required:

Theresa May would have to be replaced as Prime Minister. I don't believe she'll resign. She is under a very great deal of strain, and might have a health breakdown; more probably, there could be a vote of no confidence in her or her government.

A Vote of no confidence in May by Conservatives would require that 48 letters of no confidence from Tory MPs are sent to the Chair of the 1922 committee. The European Research Group have been trying hard to achieve this total, and have clearly failed; however, losing the vote on the 12th could change that.

But if the total were achieved, a Tory leadership election would take time, and, given that the membership of the Tory party are much more extreme Brexiters than the population as a whole, such an election is very unlikely to produce a leader who would support a second EU referendum.

A vote of no confidence in the government by parliament would require all of the opposition parties (including the DUP) to vote together. It's unlikely that all Labour members would vote against the government, but a few Conservatives might, so it's possible that this would pass. What happens next is tricky. Corbyn would have an opportunity to form a government, which would require all the parties other than the Tories to form a coalition.

Tories would need to cross the floor. Corbyn himself doesn't want a referendum; the DUP definitely don't; so such a government would be unlikely to support a referendum. The only possibility there is if some Tory MPs (possibly Sarah Wollaston, although it's hard to think of many others) crossed the floor to join 'a government of national unity', which would not include the DUP.

The EU would have to agree to an extension of Article 50, even if a government prepared to hold a referendum could be formed. The new government could not be in place before January at earliest; a referendum campaign takes considerable time; if an extension is not agreed, then the clock will run out.

The second referendum probably wouldn't change things. Polling on a second referendum indicates that the UK as a whole is still split about 50/50 within the margin of error. There has not been a decisive swing from leave to remain. Furthermore, we have not effectively addressed the problem of dark money; hostile actors, including foreign actors, would continue to successfully funnel money into the leave campaign. I think we would again see Scotland, Northern Ireland and London voting to remain, England and Wales voting to leave.

A second independence referendum has grave problems

I have been working since September 2014 for a second independence referendum. I now think
  1. It's unlikely to happen;
  2. It's unlikely to be seen as legitimate;
  3. We're unlikely to win it.
It's unlikely to happen because the SNP are too timid to even call it unless independence is well ahead in the polls, but, without a focussed campaign, we won't get there. No one but the Scottish government can effectively call one, and it will take decades to build up a new pro-independence party to the point that it can force the SNP to move out of the way.

Even if it happens, it's unlikely to be seen as legitimate. Although Keith Brown argues (and I agree) that Holyrood alone has the authority and the mandate to hold an independence referendum, a section 30 order is almost certain to be refused. The unionist parties - the Tories, Labour, and the Liberal Democrats - will use this to claim that the referendum is illegitimate, and will very probably campaign for a boycott. If the No campaign effectively boycott a referendum, the result would only be seen as legitimate if substantially more than 50% of the entire electorate voted Yes - because, in the media, all those who didn't vote would be assumed to support 'No'.

An independence referendum only succeeds if the rest of the world sees it as legitimate. If other governments - especially those in the EU - don't recognise the government of Scotland as sovereign, we've failed.

But in any case we'd be unlikely to win it. We are no longer holding the referendum from within the EU. From within the EU, while the whole aquis of European law held in Scotland, the negotiation would have been relatively easy (except fishing); and it would be preserving a large proportion of the status quo, which would give cautious voters confidence. By contrast, a second independence referendum could now happen only in the chaos of disentangling the UK from the EU, or after the Brexit dust has largely settled.

After the Brexit dust has settled, in a much poorer nation with a very recent memory of years of painful disruption, I think it would be hard to motivate any electorate to vote for major constitutional change again. Certainly it would be very easy to run a No campaign on the slogan 'no more chaos'. Also, after the dust has settled, new trade deals with the US will rapidly lead to a selloff of key state assets to American companies, and I do not see how we could protect the Scottish NHS from that. I don't believe we can afford to wait for the dust to settle.

But while Brexit is in progress, the unionists will still be able to paint a picture of the coming 'global Britain', and the 'sunny uplands' of new trade deals which will at that stage still be in fantasy land and thus capable of being painted in glowing colours. That will of necessity push the Yes campaign into a 'project fear' position, where we will have to be continually pointing out the risks - of losing the NHS, of losing our GM-free status, of losing our health, employment and environmental protections, of competing on labour rates against developing economies. Negative campaigns can win, of course, but I don't think that's a style of campaigning we'd be comfortable with.

'Business as usual' may not contine 

Human beings are programmed to believe that tomorrow will be very much like today, next year very much like this year (damn, I hope not!). We assume business as usual will continue. But if we have a no deal Brexit - which, I reiterate, I think almost inevitable - it won't. Our supply chain is very fragile. Supermarkets keep stocks of only a few days' food and groceries. Much of our food comes from Europe. Chaos at the ports will very rapidly lead to shortages, which would lead to panic buying, which would lead to empty shelves.

The British are not used to going hungry. Social order would rapidly break down. There are already plans to deploy the army on the streets.

If a State of Emergency was declared, elections would almost certainly be suspended. It's quite likely that Holyrood would also be suspended, especially if there was a significant push for independence. As senior MSPs are unlikely to consent to Holyrood being suspended, it's likely that many would be interned; it's likely that influential Yes movement figures would also be interned.

In this scenario, independence would be literally impossible without civil war, and that is not a road I want to go down. So let's assume (for now) (and devoutly pray) that this dark fantasy of mine does not happen, and go onto more optimistic options.

An 'Independence Now' manifesto would work

As Margaret Thatcher repeatedly pointed out, and as the Irish precedent shows, all that is required under the British constitution to achieve Scottish independence is for pro-independence parties to win the majority of Scottish seats at Westminster - as, actually, the SNP did at each of the last two Westminster elections and almost certainly will at the next.

Of course, at the last two election, the SNP did not stand on an 'Independence Now' manifesto. If they did, they could make every Westminster election until independence a de facto independence referendum. Of course, it's likely that the unionist parties would join forces to put up just one candidate against the SNP, but that actually plays in the Yes movement's favour - because such candidates would inevitably either be Tories or be in explicit alliance with Tories.

There are three possible outcomes:
  1. We don't win a majority of seats, in which case nothing is lost and we can try again next time;
  2. We win a majority of seats and a majority of votes, in which case we have a clear mandate to immediately negotiate independence;
  3. We win a majority of seats but not a majority of votes, in which case we'd be well advised to negotiate for maximal devolution but could still try again next time.
Consequently, I think the 'Independence Now' manifesto is now the best strategy for the movement. Obviously it would be best if the SNP were to stand on this Manifesto. But if they won't, I think that the Yes movement should think very seriously about putting 'Independence Now' candidates up against the SNP in all Scotland's constituencies.

Monday, 19 November 2018

The inflections in the path

Zoe, aged one, with her mother
I was Zoe’s wicked uncle; and I want to frame the arc of Zoe's life that lead us here, to this place; to try to share with you my understanding of it, to explain to you why I cannot mourn this death, but only the life that lead up to it.

It was a life marked by four deaths: four inflections in the path.

Zoe was a child of Auchencairn, the field of stones, between the blue line of the granite and the grey line of the sea. The primary school she went to had thirty pupils. Even in that small place, Zoe was a solitary child, often solemn, with few close friends.

She was brought up by her mother, my sister, who was bright and sociable and full of changing enthusiasms, but also ill; and by her grandmother, my mother, who was academic and intellectual and who mostly sat in her chair and read books, or wrote; and to a much lesser extent by me, with whom she had adventures and all too often ended up in deep water.

Her mother was ill, and in pain, and that was just how it was. She was brave, and rarely complained, and so when she had a persistent cough, she still didn't complain. After a year she went to hospital for it to be checked out. Thirteen days later she was dead, of cancer. Zoe was sixteen.

My mother; Zoe's grandmother
A year later, her grandmother died; and with that death, all the things which had carried Zoe through her childhood - her childhood home and the women who had raised her - were gone.

Two deaths. She was eighteen. But she did well enough in her course in her theatre course in Edinburgh to earn a place at Dartington.

At Dartington she met Rachael, who was the love of her life. But they met because Rachael, learning that she had terminal cancer, went to the other person on her course whose life, she knew, had been touched by cancer: to Zoe. In that meeting, in that falling in love, in that romance and relationship and marriage, the ending was known from the first day.

The knowledge of that ending cost Zoe. At times she denied it, but it always hung over them. They went and had adventures together - and got into deep water together - and lived life as fully as they could, knowing that their time was limited. It was not as limited, in fact, as they expected: the doctors had told Rachael she might live four years; she lived twelve.

But death came, and that death was brutal.
Rachael (centre) and Zoe (right)

Again, Zoe lost the most important person in her life, and with her, her home. Her family was reduced to two dogs, Winston, Zoe's own dog, and Lola, who had been Rachael's.

Three deaths. She was thirty four.

Zoe moved with the dogs to a new flat; not a bad flat, in itself. I came down to help her move, and to help her decorate it. She was in grief, as anyone would expect her to be; but it seemed at least she had a safe base.

Winston was a very big, powerful dog. He was not, in my opinion, a vicious or aggressive dog. But he and Zoe were close, and when Zoe was upset, he wanted to protect her; and she was upset. And so there were a series of incidents in which Winston displayed aggression towards people who, he may have thought, threatened Zoe. She became very afraid he would attack someone, and that she would not be able to control him.

Zoe with Winston
Consequently, earlier this year, she made the decision that he should be put down.

Four deaths. She was thirty five.

And yes, of course he was 'only a dog'. But he was her dog, and pretty much the only committed relationship she had left. She had let him down; she had, in a sense, betrayed him to his death. He was only a dog, but that understates the freight of guilt and grief that death carried.

Zoe died of grief. She had cause to die of grief. Most of all for Rachael, her wife; but also for those other three deaths: her grandmother, her mother, her dog.

And I cannot grieve her death. Her suffering is over. My grief is that, if her mother Jenny had not died, if her dog Winston had not died, most of all if her wife Rachael had not died, Zoe would not now be dead. Zoe would not now be struggling with mental illness. Zoe would be well, creative, inspiring, quirky, eccentric, exasperating, happy.

Do not grieve this death, for in this death there is grace. Grieve, if you must, with me, the inflections in the path.

Saturday, 27 October 2018

The Standingstone Model

The old feed store, with Alice's parrots.
All photographs in this post are from
It's no secret that I live at Standingstone. In fact, if you check at Companies House, you'll see that I am the chair of Standingstone Farm Limited. So what is Standingstone?

Standingstone is a conspiracy.

It wasn't set set up to be a model of how you solve the problem of rural depopulation, and, indeed, it doesn't, in itself, serve as a model to solve the problem of rural depopulation. But it serves as a seed from which the shape of a model can be discerned.

Standingstone is a conspiracy of homeless people to house themselves; to house themselves by buying a farm. It isn't - it couldn't be - a conspiracy of just any homeless people. To join the conspiracy, you had to be
  • Local to Auchencairn village;
  • Unable to afford to buy a house locally;
  • Unable to access social housing locally;
  • Able to afford the minimum buy-in to the project, which was £30,000, so not trivial;
  • Willing to agree the principles of sharing with the rest of the group;
  • Unconcerned by the fact that we would not have and were unlikely to get planning permission for the homes we needed.
In other words, you needed to be to some degree a scofflaw, and to have access to at least some capital.

There were some people who were otherwise qualified to join the conspiracy who couldn't reach agreement with the rest of us over sharing; there were some people who we'd really have liked to have here who couldn't raise the buy in price. Those people are not here. That may sound ruthless, but it is what we could pragmatically achieve.

It's consequently a very select group. We were also lucky that left-wing local farmers were prepared not only to offer us the farm, but to wait almost a year while we built and negotiated a group that was able to raise the price (in the end we bought only eighty-eight acres of the one hundred and twenty we were originally offered, because that's all we could afford).

But the number of people normally resident on the farm is about twenty-five, in eight households; and there's a much larger penumbra of people who are associated with the farm, who are often in practice resident. That's an increase of more than 1000% on the two people who lived here before we bought it. It wasn't set up to reverse rural depopulation, but it has the effect of holding people in the area who would otherwise have been forced to leave. And, the farm is home to more than 10% of the children in the local school. In that sense, we're actively reversing depopulation.

We operate as a sort of hybrid between a co-op and a conventional company. Each household that bought in bought a croft (originally eight crofts, now seven). The crofts vary in size between five and twelve acres, depending on how much money was put in. But each croft, regardless of size, regadless of money invested and regardless of how many people live on it, has two votes.

We own most of the woodlands, some of the pasture, and all of the agricultural buildings which were on site at the time we bought the farm, in common.

None of these crofts provides a full household income, although two provide substantially all of the household's food with a surplus to sell. But we none of use depend on our crofts for the whole of our income; we are variously software engineers, artists, musicians, tree surgeons, joiners, nurses, blacksmiths, electricians.

One household occupy the farmhouse; they had enough income to be able to raise a mortgage. The rest of us occupy old caravans, old horseboxes, a yurt and a hut. That's changing. One household has built a magnificent barn, while another household is currently building a real house with real planning permission. Both of these buildings have been built largely with sweat rather than money - the timber for both arrived on site as logs and has been sawn here. But they're both valuable.

This represents something that needs to be learned; a weakness in our model which should not be reproduced. When we negotiated the original settlement, the deal was that the company (that's all of us) had first option to buy any croft that was sold, at a price which more or less represented the purchase price of the croft. If either of these crofts were to be sold, it would be unjust for the company to say 'we can buy this croft for the original purchase price'; but it would be simply impossible for the company to buy the croft at a price which recognised the value of the buildings. If the crofts with these buildings are sold on the open market, it will be impossible for households in housing stress to buy them - they'll be by far too valuable, and consequently they'll sell to people from outside the local economy who have wealth unavailable to people in the local economy. So while we've largely solved the problem of housing stress for us in our generation, ultimately we may just be creating buildings which will become part of the problem.


The problem

View over the commons
Many remote rural areas of Scotland, especially in areas of good scenery, are now characterised by house prices which are significantly higher than the local economy will support. This means that people who have jobs in the local economy can't afford housing, and that the housing that is available sucks in people whose wealth and/or income derives from outside the local economy.

While there was at one stage a lot of optimistic prediction that the internet would bring high value work to rural areas, this hasn't really materialised. There are a number of reasons for this; one is that it is and will for the foreseeable future be cheaper to deliver high bandwidth connections in urban areas; and another is that knowledge work generally is teamwork, and teamwork does not really work as well without the constant informal interactions of proximity. This isn't to say that improved digital communications can never bring higher value jobs to rural areas, but they haven't yet.

At the same time the population of rural areas has declined steeply over the past century to the point where population densities are in many places too low to support local services. In my lifetime, Auchencairn had a soutar (cobbler), a bakery, a post office, a pub, three shops, a garage, a school, a church, a police station. Even that was a sharp decline from fifty years previously. Now, one shop, incorporating the post office, remains but its future is uncertain; the garage remains, but its future is also uncertain; the school and church remain. The rest are gone. Part of the problem is the steady increase in holding size and the steady decrease in the number of people actually engaged with the land.

There are also strong factors militating against adding value to produce in the rural economy. The creamery at Kirkcudbright has gone; milk from the Stewartry is now trucked either to Lockerbie or to creameries in the central belt. A few small producers are successfully maintaining local production: Loch Arthur's cheese production, and Cream of Galloway's ice-cream and now cheese, are leveraging good produce to provide additional jobs on the farm.

Other businesses in small towns such as Castle MacLellan Foods and West Coast Sea Products add value to produce locally and retain jobs in the local economy, but these are neither strictly rural jobs nor are they high value.

Furthermore, a bureaucratic food standards regime makes life extremely hard for small food processing businesses, as the current problems at Errington's Cheese illustrate. It seems both that Food Standards Scotland are prejudiced against unpasteurised products, and that they greatly prefer large industrial production units which are easier for them to inspect. Following the Errington's debacle, it's going to be extremely hard to raise capital for new small rural food businesses.

Agriculture does generate some high value work, of course. Agriculture uses increasing amounts of capital equipment, which must be developed, designed and manufactured somewhere. In Castle Douglas, Auchencairn's local market town, the milking machine was invented; the first milking machines in the world were designed and developed here. The company which made those first milking machines still exists; I buy plumbing parts, baler twine and tractor accessories there. But it no longer makes anything. The design and manufacture has long since gone to distant factories in distant cities, and none of the wealth generated comes back.

With the lack of high value employment locally, it's inevitable that many of the people buying the housing are past employment age: people retiring from much higher paying urban economies, selling a perhaps modest house or flat in the city to buy a retirement home in a pretty village; or perhaps people in their middle years, using wealth inherited from their parents to buy a good, solid house which will stand empty for ten months of the year.

The holiday home owners are utterly parasitic on the rural economy. Typically they buy their groceries in urban supermarkets; they have their vehicles maintained in urban garages. The only local services they buy are building and garden maintenance, and those sporadically. They contribute nothing to local civic society.

Retired folk are somewhat better. They arrive at about sixty-five; for ten years they energetically involve themselves in local activities, and during that period learn something about the locality. They buy their groceries locally, service their car locally, get their hair cut locally, use health-care services locally. For ten years, they contribute. Then, at around seventy-five they become infirm. Even then they're not entirely parasitic: they're increasingly providing employment for health and care workers - but their care is being paid for out of a local budget to which they did not contribute during their earning years. At around eighty-five they die, and are replaced by a fresh wave of energetic sixty-five year olds arriving from the cities.

But there is one significant problem with retired folk. They want the landscape of their idyll unchanged the way it's always been - by which they mean how it was ten years ago when they first saw it. They're extremely resistant to changes that could bring jobs and wealth into the community, and they're completely unaware of what the local landscape looked like just twenty years before.

You cannot build or maintain a community on the basis of a rolling population of elderly strangers, no matter how talented, well meaning and energetic. Furthermore, a population of elderly incomers provides no children to keep local schools open. Finally, of course, in a community where all the decent housing has been bought up at high prices, there is nowhere locally for the care and health workers needed to service this elderly population to live.


The shape of the solution

A family with four children under twelve live here
To achieve a population density to maintain local services, we need to be able to provide desirable homes for people of working age, with children, at a price that work available in the local economy can fund.

Why can't we?

In a capitalist market, the price of a good (in this case shelter) is the price which some person is prepared to pay for it. It doesn't have to be the person who needs the good; thus throughout the Irish potato famine, when a million people in Ireland starved to death, Ireland was exporting record amounts of food. Similarly in the Ethiopian famine of 1984, Ethiopia was generating foreign exchange by exporting water melons to Europe. It wasn't that food wasn't available; it was that the food which was available wasn't affordable to local people, because wages in the local economy didn't generate enough money. So it is with housing in rural Scotland.

The factors which drive prices up are high demand, limited supply. Supply of rural housing in Scotland is largely limited by planning restrictions, imposed by relatively distant bureaucracies. In some places it's also limited by monopolist landowners, who refuse to release land for housing. High demand is generated by the steady supply of pensioners retiring from successful careers in the cities.

We need to produce housing - good housing, housing which people will aspire to - which is protected from the capitalist market, and reserved for people who work and raise children in the local economy. At the same time it is essential to ensure those people can have a good life. To lead a good life on the restricted income which is available in a remote rural economy, you have to be able to get the best out of living in a rural place. You need to have access to your own ground, where you can grow food to supplement your diet and even your income; you need access to your own woodlot, from which you can extract carbon-neutral fuel from domestic heating.

And you need access to neighbours - to community. To people with whom your can share your joys and celebrations, but also your major work projects; so that raising a house, or gathering in a harvest, can become a communal event, a shared event, a celebration in itself.


Adapting the model

As I've implied above, there are mistakes that I believe we've made in setting up Standingstone, mistakes which will militate against it being a sustainable place for people who earn their incomes in the local economy to live beyond our own generation. Our members will, gradually over time, build legal houses with planning permission, and those houses will represent sufficient wealth that, should they or their heirs choose to sell, the rest of us will be unable to afford to buy the croft in, and anyone buying on the open market will not be the sort of people we wanted this place to be.

So to take us uncritically as a model of how to build new rural communities does not work. But relatively little change would be needed to make the model sustainable.

Rural Housing Burden

I see two ways forward. The first would be to make all houses we build subject to the Rural Housing Burden. The Rural Housing Burden is a truly estimable scheme introduced by the Scottish Government, the terms of which are as follows
"The purpose of the Rural Housing Burden title is to maintain affordability on a property in the event of future sales. The plot the title is placed on has usually been acquired at well below the market value and the title ensures that the discount is included in the future sale price. The community have 42 days in which to accept the offer. 
"If the burden is not exercised, it will lie dormant until the next sale, so it remains in the title in perpetuity."
We could have made all houses we built subject to the Rural Housing Burden. We discussed it, but we didn't agree to it. I think this was a mistake. However, a future community could make this part of their ground rules.


The alternative is that instead of selling crofts to its members, a future community would let liferents on crofts. A liferent is a traditional Scottish form of land tenure in which someone is given the usufruct of a piece of land for their lifetime. When the tenant died, the land would return to the landowner, which in this case would be the community. To adapt the liferent model to our needs now, the liferent would probably need to be given to a cohabiting household on an 'either or survivor' basis, so that if a member died, their surviving partner would still have security of tenure to the end of their life.

Of course, in modern conditions, cohabiting households do not necessarily cohabit for life. When cohabiting relationships break down, there would need to be some form of negotiation over who would continue the tenancy, but that is outside the scope of this essay.

The liferent model means, of course, that households can't build homes on their land as heritable property, and consequently probably can't get a mortgage to build a home. But the community, to which the land will return at the end of the liferent, could borrow long term against the security of the property, to finance building.

I'm promoting the liferent model rather than an ownership model because inheritance is the primary engine which drives social inequity. If we create heritable crofts, they too will ultimately become valuable and will confer unearned privilege on the children of crofters.

The liferent model doesn't, of course, mean that the former tenants children can't take over the tenancy after the death of their parents; it only means that the community as a whole gets to choose who the next tenant will be. Of course the children of the previous tenants could, if they wanted, be candidates. It's probable that many communities might have a presumption that children would take over their parents' croft. The only principle I'm trying to defend here is that it should not be automatic, should not be a right. I'd emphasise the liferent model especially if any degree of public money has gone into the intitial acquisition of land; crofters really shouldn't be able to profiteer on the back of public subsidy.

A liferent might entail a one off payment to the community at the start of the tenancy, or might entail an annual payment; that is I think a matter for the individual community.

Residency condition

Living in a rural economy is precarious. A croft does not, and will not, in itself provide a family income. Crofters and small farmers from across Scotland have had to work away from their land throughout history, to earn cash income; as mercenaries, as herring fishers, as whalers, as oil workers. I often have to work away as a software engineer. It isn't reasonable to place a condition on a crofting tenancy that the tenant must be permanently resident. At the same time it's clearly an abuse if you hold a tenancy but in practice use the croft as a holiday home, sublet it to someone else, or leave it vacant.

I think that after some period of non-residency - I would suggest five years - the community should be able to terminate the tenancy. That doesn't mean that the tenancy would automatically terminate, just that there would be a right for the community to terminate it, which would be a means of forcing the tenant to start a dialogue with the rest of the community about what they intended.

Obviously, it should also be possible for a tenant to voluntarily give up a tenancy.

The importance of commons

My cattle in Commons Meadow
When we set up Standingstone, our original view was that all the existing agricultural buildings, all the woods and part of the pasture should be common. It didn't work out like that; two crofts (including mine) ended up with small parcels of woodland. But the buildings, the largest wood, the access roads and one field have ended up as common and I think that has positive value. Common assets and common responsibilities provide something concrete to bind the community together; having common pasture means that there is a buffer of reserve grazing for when individual crofters have more beasts than their own grazing can carry, without dedicating that grazing to a particular croft permanently. We also, of course, have to discuss who will have use of it when.

One thing we don't currently have, and would undoubtedly benefit from, is the equivalent of a village hall - a communal meeting and events space, although to some extent The Void, our former high-slat cattle shed which we use as communal industrial space and store, could fulfill this function. The Void is a very significant communal asset; all the buildings we've built so far have been partially prefabricated in it, our tractors and vans are maintained in it, and each croft has a substantial area of storage in it.

Roughly 25% of all our land is common; I think that's a minimum. Three of our crofts are five acre, one (the most productive) seven acre, two ten acre and one twelve acre. I think that a five acre croft is pretty much a minimum; it will with work largely feed a family but it won't create a great deal of surplus.


Financing this

Land is cheaper when bought in larger parcels. A five hundred acre farm of good grazing/reasonable arable land is currently for sale locally for offers over a million pounds, which is to say about £2000 per acre. Such a farm would support about fifty households on five acre crofts with two hundred and fifty acres of common (or rather larger crofts with rather less common, for example seven and a half acre crofts with one hundred and twenty five acres of common). A single community of fifty households may be too big to work smoothly and it might be better to split a five hundred acre block into two communities each of twenty five households, but that's a detail.

The point is, the buy in price of a community on land at this price is twenty thousand pounds per croft. Twenty thousand pounds is less than the average annual wage in Galloway, and is thus not an impossible amount for many households to raise. So it's possible - with appropriate changes to planning law - to set up communities without public money.

However, that would buy just the land. At Standingstone, the planning authorities have in effect turned a blind eye to the fact that at present almost all of us are living ilegally, in structures which do not have and would not get planning permission. Most of us aspire to build legal housing, but speaking for myself it's unlikely that I will be able to afford to do so in my lifetime.

It's reasonable that the planning authorities should turn a blind eye, since they know that more than 5% of the adults normally resident in this parish live in illegal structures anyway, and we represent an attempt in the to long term address that problem. We also provide 10% of the children in the school. There's also a probable benefit to the authorities in that Standingstone concentrates most of the hippies and scofflaws of the parish into one place. But the most important factor in our favour is that, as long term local people, we have the support of the village and of our community council.

For privately financed communities to work, there needs to be some derrogation from planning law, either by formalising the 'blind eye' or by allowing a time window for legal structures to be built; there also, obviously, needs to be a presumption that planning consent will be given for a dwelling on a croft. However I think that such planning permission should be conditional on the Rural Housing Burden applying to any dwelling constructed.

The public interest in repopulating remote rural areas

We live in an economy which is governed primarily by and for urban areas. In this economy, remote rural areas are increasingly stressed. It actually does not serve the urban economy if remote rural areas become extensive ranches producing low-value commodity foods, interspersed by picturesque villages inhabited entirely by geriatrics. The rural areas then become a complex cost for the urban economy to bear. It's to the public good to have a self-sustaining population in remote rural areas.

What changes to public policy would be needed to support the creation of 'Standingstone model' crofting communities?

Changes to planning law

Two key changes are needed to planning law. The first is needed anyway, the second is desirable anyway but needed if no public money is to be made available to support the model.

Presumed consent to construct dwellings

Interior of one of the buildings we've built
The first change is that there should be a presumption that consent will be given for a dwelling adequate to house a family to be constructed on each croft, and that that dwelling should be subject to the Rural Housing Burden. That's vital. If people can't legally construct dwellings in which they can then legally live, then the scheme is one which only scofflaws like ourselves will be able to take up.

Planning departments must also not put too onerous conditions on houses to be built. My neighbours James and Vicky, the first of us to get planning permission to construct a legal dwelling, had to spend ten thousand pounds on architects and engineers fees to satisfy the planning and building control authorities. That's more than a third of the total I estimate I could raise to build myself a more permanent dwelling, and is the primary reason I'm deterred from even attempting it.

Partially this problem could be addressed with an 'open source' library of pre-approved house plans, which I'll discuss in more detail later.

Derrogation for temporary dwellings

My (entirely illegal) house.
If poor people spend all the money they can raise to buy land, they are unlikely to have money available to build dwellings immediately afterwards. If they were to be able to raise money to build in the mainstream mortgage market, then neither liferent nor Rural Housing Burden would be viable. Therefore, unless there is public money available, legal dwellings cannot be built quickly. Even with access to public money available, it's going to take some time for legal dwellings to be built. And in the meantime, the crofters need somewhere to live while they develop their crofts and build their legal dwellings.

In brief, the law of the land is that you can't live in temporary structures except temporarily, but you can live in a temporary structure while you're building yourself a dwelling for which you already have planning permission. Given that there must be a presumed consent to construct one dwelling on each croft, it doesn't seem unreasonable that there should be toleration for one household to live in temporary structures on each croft until that dwelling is built. There might be a time limit on this - perhaps ten years - but there needs to be some flexibility.

Changes to community right to buy

The community right to buy is a scheme which provides some public money for communities to buy land or fixed assets subject to some conditions. One of the conditions is that the 'community' is a pre-existing community of place: the people who already live in an existing settlement. This is more or less meaningless when the 'communities' are composed primarily of the non-native elderly. The non-native elderly don't have a long-term investment in an area, and won't be around to see longer term schemes coming to fruition. And, of course, in the most devastated areas of remote rural Scotland, there are no existing communities of place; the landscape is largely or wholly unpeopled.

Also, for new communities to work, they need to be communities of people who can agree ground rules by which they will live and work together. Such people will not necessarily all come from the same settlement or even from the same area. They will be new, intentional communities. New intentional communities do not currently qualify to access public money under the community right to buy.

I'm not suggesting that new intentional communities should be able to exercise the pre-emptive parts of current or future community right to buy legislation; they should be able to apply for public money only to buy land which is already on the market. The public money would not necessarily have to be a grant; it could be in whole or in part a long term low interest loan, because it is not unreasonable to require some financial contribution from crofters proportional to their earning power in the remote rural economy.

Such public money would have to be on the condition that all houses built on the land acquired should be subject to the Rural Housing Burden - otherwise this just becomes a new way for people to profiteer from speculative building.

I'd also like to see a scheme under which 'Standingstone model' communities, once they'd acquired their land, could borrow for finance to construct dwellings, and could do so on the basis that the labour would be provided by the community itself. This would not need to be public money, it's entirely possible that the financial services industry could lend given appropriate guarantees; but there would have to be some degree of legislative support for such a scheme, and some degree of public guarantee would also be welcome, since if private lenders could repossess crofts and sell them on the open market we'd be back to having a backdoor mechanism for building speculative retirement homes.

Creative Commons Licence
The fool on the hill by Simon Brooke is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License