Thursday, February 15, 2024

Otto Neurath...Vladimir Putin...and the economy of real things


Just two days ago the Hindustan Times published an article with the headline, "US Senate clears long-delayed $95 billion aid package for Ukraine, Israel and Taiwan". A long-awaited proposal to continue the massacre of unlimited numbers of Ukrainians, Palestinians and to lay the ground-work for a similar future for Taiwan seems to have seen the light of day.
However, headlines seem to be written primarily for those who read nothing more than the headline (or a tweet), for in just the second paragraph, the following is written:
"The legislation will now be submitted to the House of Representatives, which is controlled by Republicans, where there is little possibility that it would pass into law. US Speaker Mike Johnson has condemned the bill."
The Russians have said time and again that providing financial aid to Ukraine may prolong the conflict but would do nothing to change the outcome. Yet, this obsession with providing billions of dollars of aid continues. 
Such is the obsession with the magical power of money brought about by the unrelenting spread of neoliberal ideology and finance capitalism around the world, that it's hard for most people to get their heads around the fact, that money, by itself, is quite nothing.
It is the things that we can buy or rent or use with the money that is everything. There is not much use in having a lot of money during a famine when there is no rice to buy with it (just watch Satyajit Ray's classic film 'Asani Sanket' on the Bengal famine during World War 2 that claimed the lives of 5 million people).
This propaganda of money is particularly strong in India, where three decades of worshiping and trumpeting of finance capitalism has granted a god-like status to the super-rich. It matters little how the money is accumulated or what is being done with it. The mere possession of it is enough to attribute the virtues of greatness, intelligence, wisdom, talent, vision etc etc to the rich. If you are great then you will definitely be rich and if you are rich then you must be great !
However, there is not much use in releasing billions of dollars to fight a war if you neither have sufficient quantities of weapons and equipment to buy with it, nor have the industrial capacity to produce it. Already back in 2022 news articles appeared about the "donation" of 20000 rounds of 155 mm artillery ammunition to Ukraine by the US and Canada only to be followed by other news articles that would discuss the difficulties in producing and procuring sufficient quantities of shells in a short time.
And all that while, the Russian armed forces were firing about 60000 155mm shells on Ukrainian positions every single day. 
Money can buy shells if there are shells. Money cannot magically become shells. 
Apparently it was not part of the calculations of globalists that after having off-shored much of the industry to other countries to both maximise profits and break the backs of trade-unions; having set up military bases around the world to ensure that this global network stays in control; and putting in place all the financial institutions to deploy the sanction-weapons to put any mis-behaving economies in line, that they would have to see this day when they struggle to produce shells in their own countries while Saudi Arabia and Ethiopia line up to join the BRICS.
And we saw a similar situation recently with the Houthis. The peculiarly named 'Operation Prosperity Guardian' was faced with an embarrassingly prosperity-threatening prospect of having to shoot down couple of thousand dollars worth of Houthi drones with couple of million dollars worth of air defense missiles.
Already in February-March 2022, when the western powers hurled every possible economic sanction at Russia and then proceeded to freeze about 600 billion dollars of their sovereign funds, they were certain that as a result of these measures the Russian economy will tank and collapse in days.
Well, guess what ? We are in 2024 now and if this is what the Russian economy can do when it is tanked then I don't know what their economy is like when it booms. It's the sanctions that tanked, not Russia. Nobody even remembers Mr. Daleep "Sanction" Singh now.
There is no mystery to the resilience of the Russian economy. It is an economy of real things - or, as economists like Michael Hudson and experts like Andrei Martyanov have described ad nauseam - it is a REAL ECONOMY.

 


 

And this is nothing new, although it is almost always ignored by mainstream economists. Already in the 1920s the Austrian political-economist and the creator of the ISOTYPE method of pictorial statistics, Otto Neurath had discussed at length about "in-kind" accounting in economics based on his studies of war time economics. He realised that periods of crisis - such as war - clearly revealed the reality of a nation's economy and that is based on the simple fact that you can neither eat money, nor wear money, nor shoot money from money if you do not have the real goods that can satisfy all these needs in sufficient quantities.
Of course, it continues to be a mystery to scores of people in the west and also here in India. 
 
How come "rich" economies like the US and EU cannot bring a "poor" (gas-station-masquerading-as-a-country) economy like Russia to its knees with its earth-shattering sanctions ??
 
Well, as they say in many Hollywood films and series -- "Just follow the money"...and keep following it until the trail runs cold on reaching the dead site where the factories used to be.

Tuesday, November 21, 2023

Of Planning Tools and Synaptic Connections


In his last book 'Disintegration' , geo-political expert Andrei Martyanov described an interesting conversation. He wrote:
 
"Few years ago, when having a conversation with one of the former combat pilots from Russian Air Force - RuAF Officer Schools are 5 years, 6 days a week academies, same as Russian naval academies - he complained that throughout his career he never for once needed the course in Differential Equations he had to take while in the academy. 
 
The response from the group was unanimous -- they did not teach you to use Differential Equations everyday, they taught you to develop complex synaptic connections which are applicable for everyday life, including combat flying."

These lines made a strong impression on me, because they matched exactly my own experience when I first started writing scripts for solving planning tasks and then explored the applications of mathematical methods in urban planning. It is not that I particularly wanted to. My interest, experience and skills all lay in qualitative and community participation methods.
The need for that came from the real experience of having to deal with projects of the size and complexity which simply could not be tackled with qualitative and community participation methods. 
 
The most important benefit of using these techniques is not their direct effectiveness in solving planning tasks (important as that is), but the effect that their use has on the way in which we conceptualise and approach the problems that we wish to solve. 
They increase our ability to articulate complex problems clearly.

It's quite like the skills that one acquires on attending a theatre workshop. It is not important that the person actually acts in a play on stage. But the skills of communication, role-playing, improvisation, expression, dealing with an audience etc, all make a huge contribution to our day to day performance in our own professions - whatever they may be. 

The benefit is indirect - and crucial. They develop synaptic connections.
 
Exactly, like what was said by the friends of the combat pilot, the important thing is not whether mathematical methods are being used everyday to solve all kinds of planning problems, but the familiarity with the application of algebra, calculus etc to planning problems enables us to think about the problem quite differently than if we were totally ignorant of them.
How and why does that happen ?
In the last two posts we discussed the variables that effect the simple act of people going from one location within a city or a region to another location. We saw that this act of "going" is directly related to the number of people at the origin and the number of "attractors" (e.g. number of shops) at the destination; and inversely related to some power of the distance (physical distance or other barriers such as time, money etc) in between the two locations.
 
But what if we turn this problem from a relatively simple one of movement between two points to a movement between many many points ?
This is exactly the point when the practitioner trained only in qualitative skills ends up using the power of language to describe how overwhelmingly complex urban problems are. The solution to tackle that complexity is generally an open-ended call for more discussion, consultation, capacity building, participation.....and -- increasingly in present times -- for more data, as if all these processes would somehow magically resolve the problem. 
 
However, a practitioner with a more holistic education and training, would be able to do something far more powerful -- combine communication and consultation with matrix algebra !
 
If there are 20 zones within a city, then our holistic planner would make a table with 20 rows (referring to origins) and 20 columns (referring to destinations), thereby creating a "map" of numbers to describe the movement between these locations.
A more general way would be to use a variable instead of a fixed number, and say n points...instead of 10 or 20 or 50 points.
We essentially gave ourselves a highly effective "tool" that helps us to study multiple interactions using something as simple as a table with a certain number of rows and columns. Armed with this tool we are able to make far more solid and meaningful points in any community consultation process on urban mobility that we may attend.
While the equation described in the previous two articles helped us to imagine the variables affecting movement clearly; the matrix helps us to imagine the interaction between multiple locations clearly.
 
We start moving away from vagueness and confusion; and towards clarity and focus.

Similarly, when we try to algorithmise a planning task by writing a computer script, we are forced to think very clearly about the sequence of individual tasks that an overall complex problem can be broken down too.
It is not important whether the problem gets solved by the algorithm itself or not. But the very process of creating the algorithm gives us greater command over the problem we are facing.
 
The same can be said about powerful techniques of Operations Research such as linear programming. Whether we can apply it to all kinds of planning problems or not, linear programming helps us to internalise a fundamental characteristic of real planning -- the need to maximise or minimise an objective subject to a multitude of constraints.
This is extremely important, because most of this time, in complex professions like urban planning, we are not in the business of achieving a goal in totality. Most of the time we trying to optimise the situation by maximising a desirable outcome and minimising an undesirable one. And all the while we are having to do this in a context of various kinds of constraints and limitations.
 
Often, we hear planning professionals complain about the impossibility of achieving the goals set by their profession given the tremendous limitations of resources, time, capacities etc. A planner who has internalised the science of linear programming would not complain about such a situation, because to such a professional these would be the essential and "normal" attributes of the job that he or she is trained to do. 
 
Does a fisherman grumble because the fish are in the water and they swim from one point to another ?
 
And does it not make sense, then, to continue to learn the art and science of "fishing" ? 






Saturday, October 28, 2023

To Go or Not to Go --> (Urban Planning and the Distance Decay Function)

The fine art of problem articulation 

The important thing about mathematical urban models is not the mathematics itself but its application to simulate urban phenomena that we are trying to understand. Therefore, even a failed attempt at creating a model may help an urban planner understand and articulate a phenomenon with greater clarity. Trying to explain an urban phenomenon in the form of an equation compels us to cut through the fog of vagueness and confusion in our minds and seek clarity.

Try to imagine what we expressed by the equation --> A=f(B, C, 1/D) in the previous blog and then try to explain it in words instead of the equation.

We would have to say something like - "When we consider shopping or any such pattern in a city...it depends on how many people are shopping....where they are shopping...it depends also on which shopping areas are large or attractive...also we must consider which are far away or close by...its a pretty complex process....but also very basic etc etc"

While the equation used just 5 letters and 1 digit, the verbal explanation used about 250 letters (and counting). 

This is not to say that the equation is better than the description, but that one should attempt to formulate an equation from the description - if only to check if that is even possible. It is an iterative process where a description helps us to form an equation and the equation in turn helps us to give a clearer description and so on.

Distance decay and the Gravity function

Distance is a crucial topic in urban planning. As we have seen in the previous blog, "distance", in this case, is not just a physical distance, but the measure of difficulty (or ease) involved in reaching where we wish to reach -- it could be kilometers of roads; delays due to traffic jams; the high cost of petrol; the physical and emotional stress of spending hours in travel etc.

The funny thing with transportation is that it is the only land-use that does not exist for itself, but for the primary purpose of facilitating interaction among other land-uses. We are generally not on a road because we want to "go" to the road. We are on it because it links where we are (our point of origin) to where we wish to go (our destination).

The other funny thing is that distance...decays.

What this essentially means is that when distances increase between us and a particular destination, our desire to go to that destination also decreases (or decays). This is easy to imagine. Let's say that there are two cafes which are equally attractive to you. Would you rather go to the one that is 500 meters away from you or to the one which is 12 kilometers away ?  

This is what we expressed in our equation as 1/D, i.e. as D increases the A (number of trips) decreases. But by how much ?

Way back, in the 1930s, the American economist William J. Reilly discovered from his empirical studies on the flow of retail trade, that the volume of trade between cities increased in direct proportion to the population of the cities and in inverse proportion to the distance between the cities. The trade not only decreased  but it decreased in proportion to the square of the distance between cities. 

Based on these observations, Reilly formulated a law which states:

"A city will attract retail trade from a town in its surrounding territory, in direct proportion to the population size of the city and in inverse proportion to the square of the distance from the city."

Mathematically it is expressed as -

Ri = Pi / d2ki

Where Ri is the attraction of city i felt by city k; Pi is the population of city i; and dki is the distance between city i and city k. It has an uncanny similarity with Newton's law of gravitation where the attraction between two bodies also decreases in inverse proportion to the square of the distance between them. 

Reilly tested his model extensively by studying the breaking point between cities in the United States.  

If we accept Reilly's findings for now, then we have already developed our original equation further and we now have this -

Aij = f(Bi, Cj, 1/D2ij)


Nothing Super-Natural about it

It is clear from the way we constructed our equation, that there is nothing super-natural or god-given or mysterious about any of it.

We are just trying to analyse and describe how a certain kind of spatial interaction takes place.

This implies that there is nothing holy about the fact that the distance is raised to the power 2. Depending on the local context, it may be something else. In fact, empirical work over the years has shown that it tends to vary between 1.5 and 3 depending on a range of contextual factors.

However, many planning text books in India (including the venerable book on transportation planning by L.R. Kadiyali which every planning student is familiar with) continue to use distance to the power 2 without explaining that while it was derived out of extensive empirical research, that research corresponded to large cities in the United States, separated by an average distance of about 100 miles and was conducted in the 1930s.

(The fact that the equation still holds was precisely due to Reilly's empirical rigour - something that our scholars and researchers tend to shy away from most of the time.)

In reality you are free to play around with the power of D in order to check which number makes the equation simulate an observed reality best. Consider the following graph which shows how the distance decay curve would vary if we assume distance to decay if we raise the D parameter in the equation to a power of 1 or 2 or 3 -

This graph shows how quickly the likelihood of traveling a certain distance will decline if we increase the power of D in the model.

As the power increases from 1 (the green line) to 3 (red line) the tendency to travel farther declines. The red line corresponds to a reality where the tendency to travel decreases precipitously when distance increases from 0 to 3 kilometers and becomes almost nil when distance increases beyond 5 kilometers. 

Understanding the rate at which the tendency to travel declines based on increase in the distance can help us understand how appropriately or inappropriately important facilities are located with respect to each other in a city. It can also help us to concretely evaluate planning decisions aimed at locating facilities in a certain way. 

In the next blog we will play around with this equation a bit more by applying it to situations familiar to us.

Monday, October 23, 2023

Overcoming the fear of mathematics in planning education and practice

A vital contradiction in our education system

The former chief scientist of Airbus, Jean Francois Geneste said in a brilliant talk delivered at Skoltech, that when it comes to large and complex systems, 

"We can only master, what we can measure and mathematics is a discipline for measurement -- it is measurement theory."

What he said has great implications for our own field of urban and regional planning too. It is important to measure and to measure correctly, before planning decisions affecting millions of people, thousands of businesses and hundreds of hectares of land-uses of different kinds can be taken. 

Yet, precisely when there is a growing fascination with data and digital technologies, there seems to be a relatively low understanding of the role of mathematics in planning. A substantial part of the problem lies in the fear of the subject itself and the inability to apply it effectively in real situations.

We are all aware, that due to the peculiar limitations of the Indian education system, there is a rigid and unnatural separation between the sciences and the arts. 

This leads to a situation where people trained in engineering techniques are often completely devoid of an awareness of social issues and of creativity; and the people trained in humanities are often clueless when it comes to physics, mathematics etc.

Truth be told, this challenge exists in urban planning education outside India too, though, perhaps, not as severely. As Brian Field and Bryan Macgreggor noted in the preface to their book on forecasting techniques,

"...we had both come to planning from numerate first disciplines and, in planning schools at opposite ends of Britain, had independently concluded that there was an obvious gap in the literature on this particular subject."

The important word here is numerate - which means having a knowledge of mathematics and the ability to work with numbers. It is the mathematical counterpart of the word literate, which is the ability to read and write.

The complex is essentially simple 

The complex is essentially simple, because it is also a function of our learning, experience and skill. To someone who has never stepped into a kitchen, even making a cup of tea may seem like a forbiddingly complex task. However, to most people it is just a regular task -- a simple task. 

The funny thing is that mathematics seems more difficult when it is taught but it seems easier when it is applied !

When it is taught - especially in our schools - its difficulty is cranked up to meet the needs of the engineering entrance exams, whose purpose is to eliminate large numbers of students through a process of cut-throat competition. It is easy to see that this "goal" has nothing to do with solving practical problems of life and society.

Even when students master that gigantic syllabus and get extremely high grades, they may not have internalised the logic behind the topics and may fail to apply them creatively in real life situations. 

However, when one begins to study science subjects because one wants to understand and solve real life tasks, then the relevance and applicability of the topics are automatically evident and the human mind understands and internalises them faster.

Let's try to understand this using an example of a model, where we go from the simple to the complex and then realise that it's essentially simple.

Distance decay and equations that make you run away

Urban models are essentially mathematical equations that describe essential features of an urban system and can enable us to simulate and predict its behaviour.

Consider the following equation from Field and Macgregor's book (I will refer to this book  and David Foot's book on operational urban models many times in these blogs, as they are just brilliant) -

A = f(B, C, D)

This equation basically means that the variable A is a function of (that is, in some way, depends on) three other variables - B, C and D. Therefore, the value of A will change if there are changes in the values of B,C and D.

But what do A,B,C and D stand for ?

Let's assume that we want to understand how many shopping trips are made from various residential areas in the city to commercial areas. We can describe the components of our model in this way --

A = Number of trips made for shopping purposes (what we wish to find out)

B = Population of the residential area 

C = Number of shops in the commercial area

D = Distance between the residential area and commercial area.

But what to do if a city has multiple residential areas and multiple commercial areas and sometimes the residential areas are also commercial destinations and vice-versa ? We would like to express our equation in a way that shows interaction between any number of residential zones and any number of commercial zones.

We do it by introducing two more variables - i and j (where i stands for any residential zone and j stands for any commercial zone) and re-writing our equation thus -

Aij = f(Bi, Cj, Dij)

 Where -

Aij - number of shopping trips made from zone i to zone j

Bi - residential population of zone i

Cj - number of shops in zone j

Dij – distance between zone i and zone j

 

Our model represented by an equation comprising just a few alphabets can now handle a rather complex interaction of trips originating from any number of zones and terminating in any number of zones. 

We can refine the model further by considering the fact that A is directly related to B and C i.e. if B and C increase, chances are that number of shopping trips will also increase and B and C decrease then number of shopping trips would decrease. 
Furthermore, A and D are inversely related. If the shopping area is too far away (distance between i and j is too large) then the tendency to go and shop there would be low. Therefore as D increases, A would decrease. 
 
Therefore A = f(B, D) and A = f(1 / D)
 
Our model now becomes -



We now know how the number of trips would be affected based on the variables B,C and D. But how are they related exactly ? The model tells us that if distance increases from 5 km to 10 km then number of trips should decrease, but by how much exactly ? Do they interact linearly i.e. for every unit increase distance the number of trips decrease by one unit; or in some other way e.g. one unit decrease in trips when the distance becomes a square (from 5 km to 25 km) ? 

 

What we have discussed are the concepts of gravity (how strongly do different zones attract each other) and distance decay (how the intervening barriers between zones e.g. distance, cost, quality of infrastructure etc), which are key concepts in mathematical urban models.

 
When an equation just hits us out of the blue it looks difficult to understand, but when we break it down to its components and understand the logic behind their construction then it starts getting de-mystified. In fact, science is all about de-mystification. Once we get a grip of the logic then it not only does not seem complex, it actually seems very clear and basic. At that point we can begin to play around with it, understand its strengths and limitations and get comfortable with applying it.
 

Constructing mathematical models is a creative process

It is clear that what variables we choose to construct the model depend on us and on our understanding of the reality that we are attempting to study and predict. For example, attraction of commercial areas can be measured by number of shops, but it can also be measured by total floor area of shops in a zone. Distance between zones can be in the form of kilometers but it can also be in the form of the cost of covering distance or the time spent in covering it. 

It is pretty clear that one has to be extremely observant, imaginative and creative if one wants to create a model that is able to capture the essence of various urban phenomena.There are many people who may be experts in mathematics but have no understanding of urban processes or may just apply ready-made models blindly without considering how they ought to be altered or re-constructed given empirical realities.
 
As the Soviet mathematician Elena Wentzel correctly observed,
 

In the next blogs we will look deeper into the role of mathematics in planning and explore how to apply it in planning problems we encounter in our day to day professional work.






Friday, October 13, 2023

Automating Planning Tasks 2 - (Programming with GRASS + Linux)

In the first part of this blog we had discussed how the various steps of a particular planning problem (in this case the slum-proofing problem of Jaga Mission) can be articulated, algorithmised and then automated. In the second part we shall see how the specific commands in the computer program work.

The technical steps necessary for achieving the goal (given the capabilities and constraints of government organisations executing the mission) were identified and are listed below -

(a) Identify the location of existing slums 

(b) identify vacant government land parcels near the existing slums 

(c) check them for suitability 

(d) generate map outputs for further visual analysis and verification. 

It is clear that the problem solution involves spatial analysis tasks to be performed on a Geographic Information System (GIS) software. 

While QGIS is the more familiar and user-friendly option, my preferred software for tasks like this is GRASS, which stands for Geographic Resources Analysis Support System.

GRASS is an extremely powerful and versatile open-source software, which was originally developed by the US Army - Construction Engineering Research Laboratory (CERL) and then by OSGEO - The Open Source Geospatial Foundation

A very useful feature of GRASS is that when you undertake any operation on it (e.g. clipping features in one map layer using features in another map layer), the command line version of the operation is also shown.

This is because GRASS is not just a point-and-click software (software which rely on a Graphical User Interface - GUI - for performing the operations by clicking with a mouse), but it also contains a command line option.

Most computer users are unfamiliar with the Command Line Interface (CLI) - even intimidated by it (I certainly used to be not so long ago). They, therefore continue to work on the user-friendly environment of the GUI and doing all their tasks with the mouse. However, by doing that they fail to harness even a fraction of the power of these wonderful machines - there is a reason why users who work comfortably with the CLI are called Power-Users. I have written about the advantages of the CLI in earlier blogs.

When GRASS is launched, it opens three windows simultaneously - the Graphic console (for performing operations using the mouse) ; a display window (for viewing the maps and results of operations) ; and a command line window (for performing tasks by typing in commands).



An interesting thing happens when you do any GIS operation on GRASS. Let's say you wish to execute the module called v.extract , which creates a new vector map by selecting features from an existing map (quite like exporting selected features to a new map layer in QGIS). The series of steps is similar to that in QGIS, where you specify the input file name, the output file name (of the new file), any queries that specify which features are to be selected etc.

You will notice, that while you are specifying these details by clicking with the mouse and entering file names, the following line is getting typed at the bottom of the module window -


This line is basically the command line version of the v.extract operation. This is how it reads --

< v.extract input=bbsr_slum_houses@training where="ward_id = 16" output=ward_16_slum_houses --overwrite>

The command begins with the name of the module, then specifies name of input file, name of input file and the sql query (in this operation the query selects all slum houses that are located in ward 16). The optional --overwrite can be added if you run the operation multiple times and would want the previous output files to be overwritten automatically (useful during scripting/programming).

When you click the "copy" button in the module window, this command line gets copied to memory. You can then paste it in a text file to make it a part of your program.

A good practice is to run all the operations manually once using the mouse, just so that you are clear about all the operations and all the command line versions of the operations. With practice you would not need to copy the commands. You will simply be able to type them out from memory or my referring to the manual page of the respective module.

Turning individual commands to a script 

Commands are very powerful because instead of clicking with a mouse many times, you can just type a short line and get the job done. But the main power of the commands is that you can write a bunch of them down in a sequence in a text file and execute it as a program to automate all your computing tasks. 

Consider the following 4 commands written down in a sequence -->

(1) v.buffer input=centroids output=buffer type=point distance=$buffer

(2) v.clip input=orsac_tenable clip=buffer output=vacant

(3) v.out.ogr --overwrite input=vacant output=$LAYER_OUTPUT/$ulb\_vacant_$buffer\_m_buffer.gpkg format=GPKG

(4) db.out.ogr --overwrite input=vacant output=$LAYER_OUTPUT/$ulb\_vacant_$buffer\_m_buffer.csv format=CSV

Of course, it looks a bit overwhelming ! But nothing to worry, cos everything useful feels a bit overwhelming at the start. Here is what is going on -

In step 1, the v.buffer module draws a buffer of user defined distance around the centroids of slums and saves it in a file called buffer. In step 2, the v.clip module clips features from the land parcels layer using the buffer layer file. In step 3, the v.out.ogr module exports the clipped land parcels as a gpkg vector file and stores it in your chosen folder. In step 4, the db.out.ogr exports the attribute table of the clipped file (containing details of the land parcels) as a comma separated value (csv) spreadsheet file and stores it in your chosen folder.

When you run these commands as a script then all these tasks get performed automatically in this sequence. 

It is clear from the above, that you can tie together any number of commands in an appropriate sequence to handle analytical tasks of varying levels of volume and complexity. 

GRASS + BASH

Bash - an acronym for Bourne Again Shell - is basically a "shell", a program in Unix like operating systems such as Linux, which is used to communicate with the computer i.e. it takes keyboard commands and then passes them onto the operating system for execution. 

The unique thing about Bash is that it is not just a powerful method for passing commands to the operating system but also an effective programming language.

The advantage of combining GRASS with Bash is that you can take the GIS commands of GRASS and use them as it is in a Bash script without any change in syntax by adding a flag called --exec at the start of the command.

By doing that you can not only execute the GIS tasks but combine those tasks with all the other programs and the full power of your Linux computer.

For example, in the same script, you can select GIS files of a few specific cities, undertake all the operations that your need to undertake, then save them in a separate folder, extract the attribute data as csv files, undertake data analysis tasks on them, put them in some other folder, turn both the spreadsheets and the output maps into pdf files etc etc. 

I guess this is already somewhat of an overload...and I feel tired of typing too ! I will show the combination with Bash in the next blog.

The request and the reward

Using the command line and scripting with GRASS and Bash are extremely powerful, flexible and fun processes, but they do request a readiness to learn and an openness to be creative.

The rewards...well they are immeasurable.

Thursday, September 28, 2023

Data and its Non-Use

Tackling the dynamic with the static

One of the primary challenges facing the management and governance of our cities is the fact that they are extremely complex and dynamic systems. At any point of time, the decision-maker has to juggle multiple unknown and possibly unknowable variables. Dealing with our cities, therefore, must be seen as a fascinating challenge of having to deal with uncertainty.

But how does one achieve it ? How does one master the uncertain and the unknown? There are many ways of doing it, which, unfortunately remain unused and abandoned by our city planners and decision-makers. Our planners, obsessed with the preparation of voluminous master plans, often ignore something very fundamental - you cannot tackle something extremely dynamic with something extremely static.

As the renowned architect-planner Otto Koennisberger had already observed six decades ago when he was preparing the master plan of Karachi, that by the time such master plans are ready they are already out of date. In such a situation not only do the decisions of the planners hit the ground late, but the feedback is hopelessly delayed too.

The importance of feedback...and of cognitive dissonance

Anyone dealing with complex systems would be familiar with the crucial value of feedback. A system that continuously fails to correct itself in time is a doomed system.

Yet, are things like handling vast amounts of data, taking decisions in real-time, continuous course-correction based on feedback through a network of sensors really such insurmountable problems in the present times ?

On the contrary, a characteristic feature of the present times is not only a complete technological mastery over these challenges but also the increasing affordability and accessibility of such technology. Does Google maps wait for a monthly data analysis report to figure out how many people had difficulty reaching a selected destination and modify its algorithm accordingly? It shows an alternate route instantly when it senses that the user has mistakenly taken a wrong turn.

We take this feature of Google maps as much for granted as we take a 20 year perspective plan of a city for granted - such is the collective cognitive dissonance of out times.

Tackling uncertainty by shortening the data collection cycle

Let us consider a problem more serious than reaching the shopping mall successfully using google navigation. We are all aware of the havoc that flash floods cause in our cities. They are hard to anticipate because they can occur within minutes due to extremely high rainfall intensities. As a consequence of climate change we can only expect such events to turn more erratic and intense over time. While the rain happens for a short duration, the available rain gauges still measure the average rainfall over a 24 hour cycle. Therefore, despite collecting vast amounts of rainfall data, we may still not be able to use it for predicting the occurrence of flash floods.

However, in the last few years, the Indian Space Research Organisation (ISRO) and the Indian Meteorological Department (IMD) have installed Automatic Weather Stations (AWS) at various locations across the country which record rainfall data at intervals of less than an hour. Based on the data from the AWS we not only get better data sets for analysis but can respond in near real-time when the event happens.

 


In this case, we tackled the uncertainty of the rainfall event by reducing the data-collection cycle from 24 hours to less than an hour. We didn't eliminate uncertainty but we definitely limited it.

While tackling complex and uncertain situations, our main allies are not needlessly vast quantities of data but the ability to clearly articulate the problems faced and then attempting to algorithmise the problem - the more articulate the problem statement, the more effective the algorithm.

 

Algorithmising Jaga Mission tasks

Let's take another example of Jaga Mission - Government of Odisha's flagship slum-empowerment program. Jaga Mission has arguably created the most comprehensive geo-spatial database of slums in the world. Its database consists of ultra-high resolution (2 cm) drone imagery of each and every slum in the state.

But the vastness of the data by itself achieves nothing - except increasing the headache of custodians of the data who do not possess necessary data handling skills.

It is only when the variables contained in the data are identified and linked with each other, that one can feel its true power.

The database essentially consists of four main components for each of the 2919 slums of the state -

    (a) The ultra-high resolution drone imagery  
    (b) The slum boundary map layer
    (c) The slum houses map layer (containing household information)
    (d) The cadastral map layer (showing land-parcels and ownership)

These four components can be combined in myriad ways to tackle a whole range of complex problems encountered during implementation.

Does a city wish to know whether some slums lie on land belonging to the forest department in order to avoid problems during upgrading ?

No problem! Just filter the forest parcels in the cadastral layer (d) and find out exactly which households are affected by creating an intersection with the slum houses layer (c). Similarly, total area lying on forest land can be found by creating an intersection with the slum boundary layer (b).
 
Does the forest department wish to know the condition of vegetation in such slums before it allows the upgrading process? Easy. Turn on the ultra-high resolution drone image (a) and count every leaf if you wish.

Do revenue officials in cities located hundreds of kilometers from each other need the details of a specific kind of protected land parcel (let's say "gochar kissam" - grazing land) that the slums may be located on ?

Well, just create a computer program that loops through the layers of the slums of different cities and undertakes the intended operation. I showed one such program in the last blog.

The revenue inspectors would not have to run around in each and every slum of each and every city to manually verify something that is easily done using the digital data-sets.

Such a well articulated system design can not only solve its own internal problems but can also offer solutions to other related systems - for example environmental sustainability; disaster adaptation etc.

However, the mainstream approach in Jaga Mission ultimately involved turning the geo-spatial database into thousands of PDF files and paper print-outs and then proceeding to tackle these complex tasks with the brute force of manual labour.

What a data-loss !

Monday, September 25, 2023

Automating Planning Tasks - Part 1 --> (100 mb Powerpoint file Vs 3 kb text file)

What computers were not meant to do

"But in running our institutions we disregard our tools because we do not recognise what they really are. So, we use computers to process data, as if data had a right to be processed, and as if processed data were necessarily digestible and nutritious to the institution, and carry on with the incantations like so many latter-day alchemists."

- Stafford Beer, 'Designing Freedom'


The cyberneticist Stafford Beer wrote these lines in his typical humorous style way back in the 1970s.

Despite Beer's best efforts, in the years since the publication of his essays and with the tremendous increase in the processing power, storage capacity and affordability of modern computers, this obsession with data has also kept on increasing till it reached the ludicrous levels that we see today when the act of collecting of vast amounts of data itself justifies the purpose for collecting vast amounts of data.

A particularly tragic situation is one which is quite typical in the offices of the urban development sector (the field which I am most familiar with) and involves highly educated professionals spending tens of person-hours preparing graphic-heavy power-point presentations. Nothing against power-point at all ! It is a great software. The problem lies in this undue importance that professionals in the development sector feel obligated to attach to visual presentations and the time and effort they end up dedicating to the task.

Instead of making a clear presentation of the activities being undertaken by the organisation (the main purpose of a software like power-point), the making of the presentation itself becomes a big chunk of the activities being performed by the organisation. 

And these files are heavy ! Tens of mega-bytes just for making the whole thing cluttered with images, data visualisation charts, animations  etc. 

The same philosophy extends to online dash-boards and cluttered charts that urban planning graduate students in India increasingly make for their project presentations. 

Whether by design or not, the only effect such presentations have is to visually overwhelm and confuse the viewer, and not bring clarity to the topic being discussed.

We have all seen those bloated power-point files...no need to share examples of those eye-sores here.

Now let's see instead the power of a simple text file containing a script, and with a size of only 3 kilobytes.


The 3 kb text file

The following screen-shot is of a program I wrote for automating the technical steps of the slum-proofing vertical of Jaga Mission - the landmark slum land titling and upgrading initiative of the Government of Odisha.


I will explain the slum-proofing vertical in detail in another blog. In this one I will just outline the structure of the program.

The process involved certain very concrete technical steps - (a) Identify the location of existing slums (b) identify vacant government land parcels near the existing slums (c) check them for suitability (d) generate map outputs for further visual analysis and verification.

The program automates that planning process by performing the following steps -

1) selects a user-designated city from the list of total cities; 

2) draws buffers of user-designated radius length around the centroid of each slum; 

3) clips suitable land parcels (i.e. filters out categories such as waterbodies, ponds, tanks, forests etc) that fall within the buffer from the cadastral map layer containing vacant land parcels owned by the government; 

4) calculates the total vacant land available and approximate number of households that could be accommodated; 

5) outputs a report stating the total vacant land available and total residential plots that could be created on that land assuming a plot size of 30 sqm and 60 percent land coverage by residential plots.

6) outputs vector maps of the vacant land parcels for further visual scrutiny and human analysis

7) outputs maps in pdf format for a quick look by team members unfamiliar with GIS and for printing out.

It took a few seconds for this process to be completed for a city that contained about 40 slums.

If the user would like to change the city or alter the buffer distance (for example, if suitable land is not available within the buffer of the designated radius length), it can be easily done by just typing the desired inputs in the prompt asking for the city code and the buffer radius.

Considering the fact the the Mission involves 115 cities and 2919 slums, this program shortens the analytical process by orders of magnitude and allows time to be devoted to study the outputs, have discussions, refine the overall strategy and assess the probability of effective implementation.

And most importantly, writing such programs is an extremely interesting, fun and creative process. 

Have fun doing creative work and automate the rest...what could be more delightful than that ??

The size of the text file that contains this program and undertakes all these tasks in a matter of seconds is 3 kilobytes.


Is it so hard to see which one is really our friend and ally ??


To part 2...


Otto Neurath...Vladimir Putin...and the economy of real things

Just two days ago the Hindustan Times published an article with the headline, "US Senate clears long-delayed $95 billion aid package fo...