Sunday, 27 October 2013

Modelling river systems: first glimmer of success

If you've been following this blog, I've been writing about modelling a large game world for some time. Ideally I'd like that world to be largely procedural, so that I can have a lot of explorable environment without having a huge amount of painstakingly handcrafted models; but I also want it to look natural. And in making a landscape look natural, one of the first key things is to model how rain falls and how water drains.

So I've been playing with some experimental code to model this. It isn't yet doing even nearly what I want, but it's beginning to show the first hints of promise.

My assumptions are: my game area is in the northern hemisphere of it's planet, which spins eastward just as the Earth does; coriolis winds blow rain in from the west, and it falls mainly on rising slopes. From where it falls it drains downhill, and once it's joined with sufficient other water, it makes a watercourse, forming a dendral river system. Got all that? Good.

Here is a height-map - actually, the height-map I'm using for testing. A height-map is just like a contour map: darker colours have lower elevation,  with black representing sea level and white representing the highest mountain peaks.

As you can see, my continent has a mountain range running round its west and north; a large crater in its south-east corner; and a large area of fairly flat land, intended as steppe, in the centre. The expectation was that the western littoral should be wet and forested, the north and south littorals fertile, the steppe fairly arid, and that the crater should catch a certain amount of rain water - enough to make it reasonably fertile. The eastern littoral was expected to be dry.

You'll note that, as a hack on the map, there's a little black spot - literally a plughole - in the south-western corner of the steppe, to let the water out.

Now, here's the terrain map from the first sort-of successful run of the irrigate process:
In this map, white represents permanent snow. Blue represents water. Pale green represents grassland and dark green represents forest. Red represents bare rock or desert - land on which vegetation simply will not grow.

Now, let me be first to confess there's a lot wrong with this map, and where it is working it's partly for reasons of artifice. Also, there are some artefacts which I simply don't understand.

Firstly, it's too wet in the west. There are huge lakes which are not draining and I don't know why. The littoral is fairly flat (you can see this from the height-map) but lakes should not be forming on flat land. I need to understand why my drainage algorithm is not working there. Similarly, the east is too dry - it's intended to be drier, but not wholly arid. The crater had a hand-drawn valley system, and it's unsurprising that rivers are forming in those hand-drawn valleys. I don't understand what has caused the two visible dykes which extend out to see from the south-western corner of the continent. There is too strong a tendency for rivers to follow diagonals - it's a very visible artefact, and again I'm not sure why.

However, I'm really pleased with the drainage of the steppe, and on the southern littoral. It's a (good) start. I'm not yet prepared to publish the code that does this, because it is still much too messy. But I am feeling encouraged.

Having said that I'm not yet doing nearly as well as people who generate landscapes entirely procedurally without any attempt to model real physical processes.

Extraordinary renderings of Stuart London

OK, I know when I blog about game worlds it's usually procedural ones, and it's probably no secret that I'm trying to put together a toolkit which will render believable (largely) procedurally generated worlds. But to put together a believable world you have to understand how real world environments work, and these extraordinary models of London from just before the Great Fire strike me as extraordinary and inspirational.

These have been produced by teams from De Monfort University - formerly Leicester School of Art and then Leicester Polytechnic - for a British Library competition called 'Off the Map'; and they are built, literally, from contemporary maps. First, here's the entry by a team calling themselves Pudding Lane Productions:
This has a very grungy feel, similar to CD Projekt Red's beautiful Vizima from the original Witcher game; but (obviously) it's a much bigger and more complex environment than that.

Next, this one from the Optimistic Pessimists:


This is cleaner and crisper, not quite so realistic to my eye; but the flow from exterior to interior, and the interior detailing, is, again, inspirational.

And finally, the Triumphant Goat entry, which, frankly, I find most inspirational of the lot:


The actual competition was won by the first of these - Pudding Lane Productions - but they're all very good, especially in view of the quite small teams involved.

All of these models were built using the CryEngine, developed for the Crysis video game; it's a free download, although it's closed source and you have to pay royalties if you use it for any commercial purpose. As far as I can see, none of the teams have made their work available for download - which is understandable but disappointing. It would have been nice to be able to learn from them.

These maps, of course, represent a real city, and there are reasons why a video game environment should not too closely model a real city. It's awkward to navigate very narrow alleys in a video game; it's easier, in a video game environment, to get disoriented than in a real environment - because you have, in fact, many fewer spacial orientation cues.

So what I'm trying to build is algorithms to create environments which have convincing verisimilitude without being real reproductions of real world environments - but nevertheless these environments inspire me.

Saturday, 26 October 2013

'The Heresy Within' - surprisingly good dark fantasy

[This is a review of The Heresy Within, a novel by Rob J Hayes]

Let's get the basic criticisms out of the way first. This book is under-edited; it's under proof-read. There are few actual spelling errors but a lot of homonym errors, and (for me) that's irritating.

This is fantasy without a lot of magic and without a lot of monsters: fantasy, in fact, about human beings and how they interact. Which is to say, frankly, fantasy as I like it. But it's nevertheless extremely dystopian fantasy. The civilised empire (in which the narrative spends very little time) is a theocracy where orthodoxy is policed by an all powerful inquisition; what precisely constitutes heresy isn't clear, at least from this book, but it has to do with the use of magic. Which is odd, because the inquisition are, for most of the narrative, the only people with the power to use magic. The principle protagonist is an officer of this inquisition.

The bulk of the narrative, however, is set in 'the wild', a very large area of lawless steppe nominally ruled by nine aristocratic families. In this lawless, chaotic and wartorn region, outlawry is rife. The novel follows a band of outlaws who ally with the protagonist to help him complete his quest - they have a contract to assassinate the aristocrat whom he has been sent to question.

Most first books of new trilogies spend a lot of time world building. In this book the world building is done very lightly, in the course of the narrative, never interrupting the flow; and it's done by showing, not telling. We learn only broad brushstrokes about parts of the world this narrative doesn't visit, and we learn them en passant; the parts the narrative does visit we see as the protagonists encounter them, through their eyes and from their viewpoint.

Characters are strongly drawn, and, despite the fact that all the key characters have profound moral flaws, are interesting and engaging; as a reader it's not hard to engage with them, ride with them, root for them. The magical physics are well explained and consistent, and although there is a mild Deus ex Machina moment, it is consistent with magical physics which has already been established.


All in all this is definitely writing on a par with authors like, for example, Joe Abercrombie, who have been much more extensively promoted. I recommend it!

Tuesday, 22 October 2013

Draft submission to the English parliament's enquiry anent Scottish land reform

Summary

In this submission I seek to argue that land in Scotland is grossly inequitably distributed, which is a public policy problem in its own right; and that, partly as a consequence of the inequitable distribution of land, our uplands have been catastrophically mismanaged, leading to a nexus of other public policy concerns. I seek to show that a progressive land tax would significantly address the first of these problems and should contribute to remedy of the remainder. However, some change in planning law is also required.

Introduction: on the basis for private land ownership

The lands which now comprise Scotland did not come into existence private. No god gave out property deeds graven on tablets of stone. Rather, over the past four thousand years, successive peoples have come into Scotland and taken land more or less by force from its previous occupiers. There can be no square inch of Scotland now, which has been passed down peacefully within the family from generation to generation from its original settlers. On the contrary, all land in Scotland has changed hands by murder, theft, extortion or deceit, most of it many times. There is no land-holding in Scotland now which is not based at some remove on malfeasance.

The most iniquitous malfeasance has been in the enclosure of commons. Until the eighteenth century, most of Scotland's land was not enclosed; rather, local people had rights of common across it. The legal theory under which land 'owners' were able to arbitrarily extinguish those rights was, to say the least of it, novel. In this part of Scotland at least, the enclosure was fiercely resisted by 'levellers', which is to say, the dispossessed majority. Enclosure progressed across Scotland slowly, and by the time the far north west was enclosed, the public mood had begun to swing against the 'right' of enclosure. So instead of 'the Highland Enclosures' we now speak of 'the Highland Clearances', but in fact the process, whether called 'clearance' or called 'enclosure', has been the same. The people of Galloway were just as deprived of ancient rights and livelihood, just as driven off the land, as the people of Caithness. It's just, for us, the process happened earlier.

Having said this, land in Scotland is (by and large) now held privately and that is established political fact. While one may argue about the outcomes for which farming is optimised, by and large lowland Scotland is farmed productively. A revolution which made our farming significantly less productive would not be a good thing.

However, ownership of land is both a good and a monopoly. That it is a good is demonstrable in that people both desire it and defend it; that it is a monopoly is simply demonstrated in that if one person owns a piece of land, another cannot also own it. Goods, and especially monopolies, are not normally subsidised. Goods, and especially monopolies, are normally taxed. There is no obvious reason why land should be different.

Finally, land is a critical national asset. It is in the interests of the community of Scotland that the community has effective oversight of the management of that asset.

Natural justice

Birthright: land area per head

Scotland has a land area of 78,000 square kilometres and a population of five million, which equates to 67 people per square kilometre or one and a half hectares per person. In natural justice it seems hard to assert that one person has more right to the enjoyment of the land than any other. Yet Richard Walter John Montagu Douglas Scott monopolises 111,369 hectares, arrogates the birthright of 74,000 Scots. The landed lobby complain that redistribution of land would be 'theft' which, they claim, would violate their 'right to property'. But, as I shall argue below, redistribution can be achieved not by compulsory purchase but by the simple processes of ordinary taxation. Land which the owners voluntarily vacate because the burden of taxation is too onerous is not stolen, it is given up.

In any case, however redistribution is achieved, achieved it must be. As this shocking graph produced by Andy Wightman from figures supplied by the Office of National Statistics shows, 1% of the population own more property by value than the other 99% put together. A community built on such gross inequity cannot stand.

House prices and the local economy

Much has been made of the depopulation and social disruption that has afflicted the 'crofting areas' - the Highlands and Islands. I do not wish to minimise that in the least. But depopulation and social disruption has affected areas of Galloway on an equally devastating scale. My home village of Auchencairn had, in the 1880 census, around 2400 inhabitants; it now has 180. That raw figure sounds devastating, and it is; but the truth is worse. Of the eighty or so stone built houses which make up the core of the village, only three are now occupied by people born in the village, and they are all in their seventies and eighties. The price of even a modest house is now of the order of two hundred thousand pounds, and local wages are depressed: the average working wage in the region is £20,800. So it simply isn't possible for people earning their living in the local economy to buy a house.

The reason for this is that people retiring from the cities, with the proceeds of selling an urban house, can outbid local people, and have done so. Most of these people are resident - only about 10% of houses stand empty - and many of them make a positive contribution to the community. But nevertheless their interests are different from the interests of people who need to earn a living in the local economy, and that does skew communal decision making away from economic development.

Furthermore, with the local private housing monopolised by incomers, what 'native' - second or third generation, or more - villagers remain are corralled into a bantustan of 'social housing' on the other side of the burn, leading to a community divided between white settlers in the increasingly gentrified village and natives in the social housing. This is not a good thing.

And finally, it's even worse than that. The people of my generation born in the village who remain in the village are the children of farmers who inherited the family farm, and those families who have subsisted largely on social benefits, and who therefore qualify for social housing. There's virtually no-one in between. Everyone else went away to university and never came back. There are no jobs for them, and there is no housing they can afford.

I'm painting a bleak picture. Driving through, you'd see a pretty village, and a vibrant village, with its own newly built, community owned post office, with its recently renovated village hall, with new community allotments, with its generally well kept and pretty houses. It's true that, by comparison with many other villages in Galloway, Auchencairn is doing well. Nevertheless, my picture is also a true one, and it is a picture of depopulation and social disruption every bit as bad as that experienced in the West Highlands. It's just less visible.

Auchencairn, of course, is by no means unique. In villages throughout remote rural Scotland this pattern is repeated. A second clearance is being perpetrated; local people with real knowledge of and investment in the locality and its landscape are being progressively driven out, to be replaced by a population of incomers who, because of their age when they arrive, are active and effective members of the community for only a few years before being replaced in their turn with new strangers. These are not social conditions in which community can be sustained.

Subsidy of the rich by the poor

It is an accepted fact that the primary cause of the French revolution was the subsidy of the very rich by the poor and middle classes. Yet we are repeating this mistake, generally throughout modern British society but especially in remote rural areas. For example, the subsidy granted to the owner of the Auch and Inchmurran estate, currently for sale, is stated by the selling agents to amount to £12,000 every single week - two thirds of a million pounds every year. That equates to the income of thirty ordinary rural folk working full time and paying their due income tax; or, alternatively, to the entire income tax take from one hundred and fifty of those ordinary hard working rural folk. There is no way this can be considered either equitable or tolerable.

Furthermore, in addition to subsidies received directly from the state, the landowning rich benefit from many other subsidies paid indirectly by the poor. They receive payments for electricity transmission across the land and from wind turbines erected on the land, which come ultimately from the energy bills of ordinary folk. And this is particularly galling, because the ownership of the hilltops on which many turbines are erected is particularly hard, historically and legally, to justify. This is, largely, stolen common land, stolen from those very ordinary folk whose energy bills are now inflated.

Holding Size

Quite apart from the issue of large estates, the average agricultural holding in Scotland is very large, much larger than comparable countries in Europe. The average holding in Scotland is 101 hectares, as compared to 62.9 in Denmark, 43.1 in Sweden, 25.9 in the Netherlands or 21.6 in Norway.
TerritoryAverage holding size (Ha)SourceNotes
USA181US Department of AgricultureQuoted as 449 acres
Czech Republic152.4European Commission
Scotland101Scottish Government2010 data
UK70.8European Commission2007 data
Denmark62.9European Commission
Germany55.8European Commission
Sweden43.1European Commission
Netherlands25.9European Commission
Norway21.6European Commission
Italy7.9European Commission
The smaller the average holding, the more families are settled on the land, and consequently the more vibrant the local communities. If a farmer in Norway or the Netherlands can feed a family on fewer than thirty hectares, a farmer in Scotland can do so as well. While the farming lobby will doubtless argue that they cannot maintain the lifestyle to which they have become accustomed on such small holdings, their interests should not be the only concern.

I believe that there is a public interest in more people having an active engagement with the land. I believe there are many people who would like to own and manage their own land if they could afford to do so, and that many of them could do so very productively.

Conservation and biodiversity

Forest cover and species poverty

No part of Scotland is naturally treeless. The natural treeline is above the summits of all but the very highest hills - boreal forest in Norway extends to above 1200 metres in altitude. There are no natural moorlands in Scotland, and extremely few natural bare hilltops. There is nothing 'natural' about a grouse moor: it is just as much a man-made environment as an open-cast mine - and almost as depleted of wildlife. To quote Shaw and Thomson 
"wooded habitats in the Cairngorms are about 13 times richer than heather moorland and 11 times richer than grassland, in terms of nationally important species. These disparities are even more pronounced when the extent of each habitat is considered. Despite being the main habitat for some 39% of important species, woodlands cover only about 17% of the land area of the Cairngorms. In contrast, moorland appears to support only 3% of the Cairngorms’ important species, but covers some 42% of its area."
The stripping of forest is partly due to unsustainable logging in earlier centuries, but is mainly due to the hunting to extinction of all our natural climax predators, leading to uncontrolled expansion of herbivore populations particularly deer - and, of course, greatly aggravated by the grazing of sheep. Deer and sheep, uncontrolled, will graze out all shoots and saplings, preventing the natural regeneration of forest and leading to total deforestation in only two or three centuries.

Without the durable and deep root systems of forests, fertile topsoil is easily scoured off steep hillsides, leading to impoverished soils and open screes which will not sustain even sheep. Our overgrazed, barren uplands are a man-made wet desert, a testament to generations of catastrophic land management.

Environment, global warming and sustainability

There's another problem with overgrazing our uplands. All that good, fertile topsoil that's scoured off the slopes and hilltops has to go somewhere, and it does. The rich organic matter which supported life on the high slopes is lost to the sea, and the mineral part - the gravels and sands and silts - are laid down in beds of our streams and rivers and estuaries, gradually strangling them.

Flooding

And in winter, now there is no longer forest and moss on the high lands to hold and buffer the rain, to release it slowly through the turning of the seasons, all that good revivifying rain has to go somewhere, and it does, too: rushing uncontrolled down those fragile hillsides into our choked rivers, and then, with an impatient shrug, rolling on over our flood defences into our homes and workplaces. Floods are not natural disasters. They're man made disasters. They're made by deforesting hillsides just as surely as they're made by building on floodplains.

Landslips

But there's another man-made disaster that's made by overgrazing hillsides. As the earth is scoured off the hillside, everything that is built on that earth is scoured off with it. The road builders of the Rest and be Thankful can never rest, whether thankfully or not, while the sheep are killing the plants whose roots once bound the hillside.

Biomass/carbon capture and storage

And there's a third issue, which is carbon. Millions of pounds are being invested in developing technologies for carbon capture and storage, but they don't work yet. Meantime, the Earth has its own mechanisms for carbon capture and storage. Principal among them are the ones called 'trees' and 'bogs'. The deforestation of Scotland's hills has released two trillion metric tons of carbon, and that's before we count the bogs that we've lost (30,000 tons per Km^2 x 78,000 Km^2 x 83% (because 27% is still forest) = 1942200000 tons). Or to count it another way, the forest we've lost - just the forest - is equal to 23 years of our current total carbon footprint.

Reforesting our hillsides and uplands would recapture much of that - far more than pumping the output from Longannet under the North Sea ever could - and at the same time those forests could be managed sustainably for fuelwood, providing carbon neutral heating for very large numbers of homes, or burned in biomass power stations to generate electricity (and, hopefully, integrated local area heating).

Remedy

I believe that a great deal of the remedy both to the problem of the inequitable distribution of property and to the ecological catastrophe of our uplands can be achieved through simple measures: the removal of subsidies, and the imposition of tax. Obviously, nothing in life is this simple. There will inevitably be need to be policy tweaks. But in this section I intend to argue that land tax is the place to start.

Flat land tax vs land value tax

The first thing to say is that while the idea of a land tax is undoubtedly good, the right land tax is better. At present, in Scotland, it is proponents of land reform who are promoting land value tax. And they shouldn't be; it is at best a distraction, at worst actively counter-productive. Land should be taxed. Ownership of land is a good, and it's a monopoly, since only one owner can own any piece of land. Land owners, just by owning land, exclude others from the full enjoyment of that land. Of course it should be taxed. But land value tax goes about it in precisely - diametrically - the wrong way.

Land value tax does not address the issues of the inequitable distribution of land, nor of the exploitation of marginal land. The land value tax on ten thousand hectares of remote highland estate is precisely nil. There is no incentive provided by the land value tax for the estate owner to divest themselves, to break up their estate into smaller holdings. Land value tax essentially subisidises grouse moors and deer 'forests'.

Land value tax on a thousand hectares of lowland arable is the same whether that land is held as one holding or as ten; but as one holding, there will be economies of scale, so again the land value tax militates against the breaking up of larger holdings into smaller ones.

Land value tax discourages the use of the land on which the public has invested infrastructure. If oil refineries are to be built, it's in the public interest that they should be built near deep water ports and railways. If factories and warehouses are to be built it's in the public interest that they should be built near transport infrastructure. If the land which is appropriately serviced for industrial development is so taxed that there's no incentive for industrialists to use it, they'll build their factories elsewhere; which means the public expenditure on infrastructure will be wasted, and the overall efficiency of the economy will be reduced, benefiting no-one.

But it's worse than that. Under land value tax, valley lands, closer to public roads, will be taxed more, and hill lands taxed less, giving an incentive to farmers to move from lands which are suitable and convenient for grazing, but are taxed, to lands which are less suitable and less convenient but are not. In other words, land value tax will inevitably contribute to the further over-exploitation of marginal lands.

Finally, Land Value Tax does not tax land. Nor does it tax the value of land. It taxes a notional, uncertain element of the value of only some land - it imposes no tax on marginal lands distant from public infrastructure. And because that element of the value which is to be taxed is notional, it requires sophisticated assessment and is open to contest by people who by definition have access to the best, most devious accountants and lawyers in the land. In short, it would be an extraordinarily expensive and inefficient tax to collect, and would be the cause of interminable court-room battles fought at cost to the public purse.

So is there a better solution? Of course there is. A flat land tax - the same tax levied on every hectare of Scotland - would make it uneconomic to own marginal land.

Progressive land tax

However, the flat land tax does not address the issue of large holdings. Fortunately, income tax already provides us with a model for how to deal with this. If you have a small amount of income, you currently pay a small proportion of tax on it. If you have larger income, you pay proportionately more. The same principle should apply to land tax: a larger holding should be taxed more per hectare than a smaller one. I've suggested, as a mechanism for this, an exponential land tax, because I think having a continuous scale (rather than the stepped one we currently have with income tax) produces fewer quirks and anomalies. But in any case, if larger holdings are proportionally more heavily taxed than smaller ones, there's a positive fiscal incentive to break up larger holdings into smaller units, and that's precisely what land reform ought to be setting out to achieve.

Explicitly, the progressive scaling of land tax should be aimed at making the ownership of large holdings, especially of marginal land, unaffordable.

All holdings with common beneficial ownership taxed as one

What large landowners, faced with this scheme, will do, of course, is to set up hundreds of separate limited companies each holding a small amount of land, and the scheme will not work unless this evasion is explicitly designed around. Therefore, two or more holdings which are found to have substantially the same beneficial owner will be counted for the purposes of the exponential land tax as the same holding. This must be so whether or not the holdings are contiguous; every beneficial owner must be taxed on the entirety of their holding. Where a land-owner, through anonymous overseas trusts or however, manages to evade tax for a number of years, the tax authorities must be empowered to levy back tax when the evasion is discovered. As the back tax on large holdings would bankrupt any entity on Earth, no-one is going to take the risk of doing this, so avoidance should not be very much of a problem.

Title defaults to common

Obviously, the question arises what happens if the land tax is not paid. My answer is this: after a reasonable grace period - say a year - title in the land on which tax is unpaid should lapse, and the land should revert to common. Title in buildings and other permanent structures on land which reverts to common should also revert to common.

It should be open to the landowner to negotiate with the tax authority, to say 'I will pay tax on only this part of my lands, and allow the remainder to revert'. But of course one consequence of that in these days of wind turbines and similar infrastructure is that landowners will seek to pay tax on only ten square metres around the base of each turbine, so it ought to be open to the tax authority to refuse such an offer.

Benefits from land should be local

So who - what public body - should benefit from the proceeds of the land value tax, and who should manage the common lands?

Size of local government areas

To answer that I'd like to make reference to the size of Scottish 'local' government areas. They are by far the largest in the Western world. They're (by population) on average thirty-six times the size of average Icelandic local government units; twelve times the size of Norwegian ones; four times the size of Swedish or Dutch; three times the size of Danish. Or, startlingly, eighty four times the size of French communes. This isn't local government at all. Fully one sixth of the independent countries in the world have smaller populations than Dumfries and Galloway.

So I would propose that land tax should be levied - by a central authority, perhaps - on behalf of community councils. Where tax is unpaid and land reverts to common, the common lands should be managed by the local community council, or, if the local community council is unable or unwilling, by a Common Lands Factor on behalf of the community council. This would, of course, imply a significant switch in discretionary public money from the existing local authorities to community councils, and inevitably some or all of the duties and responsibilities of the local authorities should switch too. That is a reform which is separate from land reform, of course, and one which should be considered separately- but it is one which I believe would benefit much of Scotland.

Of course it is by no means inevitable that local community councils would choose to reforest their upland commons; they might even choose to manage them as common grazing or for deer stalking, which would not improve the ecological situation in the least (although it is now in most places so bad that it could not, at least, be made much worse). To a degree this is a matter of local democracy: local people ought to be empowered to make their own decisions, even if to outsiders these look wrong. However, central government might seek to advise and encourage community councils to reforest.

Planning policy

Redistribution of land into smaller holdings is of no benefit if people cannot live on the land. The issue of house pricing is a very thorny one, since people with money are mobile. To ban urban people from retiring to the countryside would make a mockery of a free society, but as long as they do, with a fixed housing stock, it's inevitable that they will continue to raise the price of rural housing to a level which the rural economy cannot sustain.

Planning policy must be revised to address this problem. The Rural Housing Burden goes a considerable way towards this. However, the size of an agricultural holding on which a dwelling for the family of the owner of the property can be built also needs to be substantially reduced, although it is reasonable to impose the condition (as is frequently done now) that the dwelling should not be sold separately from the holding.

Monday, 14 October 2013

Modelling rural to urban, take two

Kirkcudbright high street, showing development of closes
and pends on runrig.
In an earlier post, I wrote about modelling the change from rural to urban, and I did so in the context of what was a grid based landscape. And that really does not work for me, because artefacts of the grid are bound to show up in the final settled landscape model, and it won't look natural. So although for height-maps I may continue to have an underlying grid - mainly because there's so much code of other people's I can re-use - I'm going back to thinking that for rivers, roads and boundaries and for point features like buildings I need to overlay what is essentially a vector map.

So, going back to populating a game world, the first generation of settlers wander the landscape moderately randomly until they find good farmland. Then they reserve themselves a long quadrilateral, not necessarily perfectly rectangular. The width of that quadrilateral is the approximate width of two building units. Why? It will become clear. The length will be approximately eight to ten building units. The quadrilateral, like a div in HTML, will be laid out with a certain padding, more at the ends, less at the sides, into which other plots may not intrude. The padding at the ends of two facing plots will naturally become a street - actors will traverse it because they may not traverse actual plots. Similarly, padding to the sides of plots will naturally become closes, wynds or alleys.

Settlers will prefer to settle near existing settlements. So a second settler will reserve a similar long quadrilateral either facing or alongside the first, and so on. This gives us a runrig settlement. I was worried about the position of buildings for a while, but I've seen a solution: one does not actually assign building positions until the landscape is rendered; instead, you assign plots 'building scores' - simple numerical values - which are incremented as the occupant of the plot gets more prosperous over time.

When, finally, the landscape is rendered, buildings are constructed as follows. Each plot constructs one building unit at a plot end adjacent to a street. Then, iteratively, for as long as there are remaining building scores, each plot which has remaining building score constructs a new building unit and decrements its building score.

The rules for positioning new units are as follows:
  • Street frontage (at either end of the plot, provided there is a street there) is used first. Recall that a plot is the approximate width of two units.
  • If a building is not already taller than its immediate neighbours, a unit may be added 'upstairs' (up to an absolute limit of N stories, where N is probably four).
  • Where an upstairs unit is on a street frontage and adjacent to a building on a different plot (long side to long side), the unit will extend over the side padding to form a pend.
  • If it's impossible to build up because neighbouring buildings are not yet tall enough, a new ground floor unit will be built behind the street frontage, alongside the close. If one neighbouring plot already has a building on a close, that side of the plot will be favoured.
  • Buildings on a plot behind the street frontage will always be on the same side of the plot.
  • Buildings on a plot behind the street frontage will never be taller than the buildings on that plot on the street frontage.
The rendering of building units will be as described in my note settling a game world - i.e. the materials and details will be related to the local geography, and the materials which would be 'naturally available' in it.

This means that for any plot we need only store the vertices of the plot, and the building score. The first vertice of the plot provided the seed for a random number generator which informs the construction of the building, guaranteeing that it will always be rendered the same. If the economy model continues to be run while the game is in progress, a prosperous non-player character may add additional building units to their plot during the course of the game, but the procedural algorithm sketched above ought to make this natural.

There's an implication that, to allow for non-player characters becoming less prosperous, whether due to action of the market or war or pestilence, buildings ought to be able to fall into ruin, so for each cell model of a 'complete' building there ought to be at least two models of that cell in states of dilapidation, and some form of 'negative building score' to drive that; but this is getting a long way ahead of myself.

Sunday, 6 October 2013

The attack burden

I have, it won't surprise you to know, a server out there on the Internet. And occasionally I do a little sweep to check its security and general health, because it is out there on the Internet, and the Internet is a pretty nasty place with lots of pretty nasty people.

Now, on my server there is only one valid login. That login requires a certificate - you cannot login with a password alone. Even once you have logged in, you cannot do anything significant in the account to which you have logged in, unless you also know the root password (although, of course, once you have logged in, brute-force guessing of the root password could be attempted). So I'm reasonably safe from attacks such as the Hail Mary Cloud.

But two things are interesting. The first is the sheer number of attempted logins. Last week I had 8322, or an average of 49 an hour, or not far shy of one a minute.

The top twenty usernames tried were as follows:

root@goldsmith:/var/log# grep 'Invalid user' auth.log* | awk '{print $8}' | sort | uniq -c | sort -rn | head -20
    185 oracle
    130 web
    123 test
     88 admin
     75 tester
     70 testing
     69 testuser
     62 test1
     56 ftpuser
     54 userftp
     51 test3
     51 test2
     50 test4
     48 test123
     40 user
     37 guest
     35 webmaster
     27 administrator
     25 info
     23 minecraft

One assumes (at least, this one does) that the account names most often tried are the names the attackers most often find give them useful access to interesting things. Which argues that Oracle installations tend to have significantly weak passwords. But notice that many systems appear to be vulnerable to logins on the 'web' user account. Surely, for any sakes, if you have such an account it ought not to be login enabled (and ought to have very minimal privilege anyway)? The fact that that's an attack target is worrying. That 'admin' and 'administrator' are attack targets isn't surprising - all sorts of systems, including many small appliances based on either Linux or *BSD, have 'admin' accounts with substantial privilege, and a lot of people coming from Windows backgrounds use 'administrator' as the name of a significant account.

'ftpuser' isn't so worrying. Yes, a lot of machines will have them, but they should be able to do little.

Which leaves the remarkable prevalence of variants on 'test'. In fact, if you look at all variants on test, there are almost 100 attempts a day:

root@goldsmith:/var/log# grep 'Invalid user' auth.log* | awk '{print $8}' | grep test | wc -l
681

So the next question is, how often was 'root' tried? After all, root is the account which you have to break in order to get total control of a UN*X machine. Five years ago, it would have been far and away the most common target. The answer now?

root@goldsmith:/var/log# grep 'Invalid user' auth.log* | awk '{print $8}' | sort | uniq -c | grep root
      1 anonftproot
      3 cvsroot
      1 ftproot
      1 nfsroot
      1 root001
      1 root01
      1 root02
      1 rootkloots
      1 sroot
      1 testroot
      1 vroot
      1 webroot
      1 wwwroot

Never. Not even once. Which is sort-of good news, and implies that people who keep UN*X machines out there in netland have learned the lesson of not allowing root login.

So, next question is, am I seeing distributed, Hail Mary Cloud style attacks? The answer is I'm not. Of my 8322 probes last week, 89% came from a single host:

root@goldsmith:/var/log# grep 'Invalid user' auth.log.1 | awk '{print $10}' | sort | uniq -c | sort -rn | head -10
   7445 190.52.65.170
    424 223.4.208.188
    259 218.108.85.245
     67 199.187.123.84
     47 46.165.220.24
     43 36.39.246.121
     12 
      7 59.124.124.9
      5 218.108.169.180
      4 112.78.3.234

So, where is that host?

root@goldsmith:/var/log# dig -x 190.52.65.170

; <<>> DiG 9.7.3 <<>> -x 190.52.65.170
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- 34849="" font="" id:="" noerror="" opcode:="" query="" status:="">
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0

;; QUESTION SECTION:
;170.65.52.190.in-addr.arpa. IN PTR

;; ANSWER SECTION:
170.65.52.190.in-addr.arpa. 43200 IN PTR static-host65N170.sls.espoltel.net.

and who's that?

root@engraver:/var/log# whois espoltel.net
Registrant:
ESPOLTEL S.A.
   ESPOL CAMPUS GUSTAVO GALINDO KM 30.5
   PERIMETRAL
   Guayaquil, Guayas 99999
   EC

   Domain Name: ESPOLTEL.NET

So my very prolific attacker is a machine hosted by (but not necessarily operated by) a small ISP in Ecuador. The machine hosts the website of the local water company. It is almost certainly a machine which has itself been successfully attacked and compromised. But there's nothing specifically Ecuadorian about the attack; my second most prolific attacker is in Hong Kong.

The point of this post is that if you host a server on the Internet, or any other device attached to the Internet, it is being attacked, all the time. Most of those attacks are fairly simple-minded. There have been no attacks I'm not familiar with, and no attacks which I think have any chance to succeed.

But the morals of this story is, first, if you have an internet-facing device which is secured only by password, don't. It will be compromised sooner rather than later. Client certificates are very much more secure. And second, keep wary. Sooner or later, a new and more effective attack will come along.

Tuesday, 1 October 2013

Of pigeons, and long distance messaging in a game world

I've written long ago about the flow of news in a large game world. Person to person news spreads slowly, imperfectly, unevenly. You can outrun it. Most current large game worlds don't have organically flowing news at all. When non-player characters pass on news relevant to the progress of a plot, it's not because news has spread to them by the natural process of character talking to character, it's because the developer has intervened and scripted it. And, of course, in games I want to write, I still want the developer to be able to intervene and script such things - where it's essential to the plot that that character should have that news.

However, there's more to the spread of information than news. Sometimes the player character must communicate with allies - either other players or non-player character allies - who are distant. Most current generation role playing games would introduce some magical device - a scrying mirror or whatever - which allows instant communication. But I want to radically limit the use of magic in my games, simply because an incoherent magical physics leads to an incoherent world, while any coherent magical physics which is more than trivial tends to ruin plots. The god should be lowered out of the machine only rarely, and never in plot-destroying ways.

So, messaging. How would real bronze-age or iron age heroes have communicated with distant allies? Well, one way is to send a messenger. That means hiring a non-player-character messenger that the plot gives you reason to trust. You've got to find that messenger. The round trip time is just the round trip time for a journey over the distance, which is to say thirty kilometres per day if the messenger is on foot, about 120 kilometres a day if on horseback, about 160 if by sea with a fair wind. Obviously the cost of the hire has to be enough to pay the messenger a premium over what he could earn from his normal occupation over that period, and enough in addition to cover the expenses of the trip. So it's quite slow and quite expensive. And if you want a message back you've got to agree a rendezvous point with the messenger (or a poste restante, possibly an inn), and you've got to actually get there. The messenger may also be unreliable, either in failing to deliver the message or being suborned by an opposing faction.

The alternative is pigeons. Pigeons will return to their home loft. Pigeons are quite light, and can be carried for a time in quite a small cage. So a hero might carry two pigeons in his pack, or four on his horse, or up to perhaps a dozen on a packhorse, in addition to normal equipment, without adding excessive weight or bulk. However, the pigeons could do only one thing: take a message to their home loft.

To get a message back - even just confirmation that the message had arrived successfully - the character sending the message would have to be somewhere which had a pigeon loft, and, additionally, the loft to which the pigeon had been sent would have to have a pigeon in stock whose home loft was where the player was. Presumably allied centres which had pigeon lofts would exchange pigeons regularly for precisely this reason, and perhaps you would have a class of non-player-character whose job was just to trundle round between allied pigeon lofts with a horse and cart (or possibly boat), exchanging pigeons.

Of course, the pigeons would not always arrive. They could be lost for a number of reasons, including but not limited to hawking by opposing factions. The probability of a pigeon going missing should probably be a function of the square root of the distance (or some other power root, but two is approximately right). The probability of a message falling into the hands of an opposing faction (assuming there is an opposing faction at this point in the plot, which if a player is sending a long distance message there almost certainly is), is probably about half the probability of a pigeon going missing in the first place.

The possibility of pigeons getting lost and messengers being suborned, of course, puts a premium on in-game cryptography, making it a valuable skill for players to learn.

But I think slow, unreliable long distance messaging would add considerably to a number of possible plots, and it would be easy both to explain in narrative terms and to implement.

Creative Commons Licence
The fool on the hill by Simon Brooke is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License