(Smart when you create...dumb when you comsume) <-- How to recognise unnecessary technology and some Jaga Mission Stories

The challenge of humanity, since the industrial revolution, has not been one of scarcity, but one of excess (and of the exploitation and inequality that naturally appear when that excess - which can provide for the whole world - ends up being controlled by the few). 

The trouble with technology is the same. How much is enough ? Where does one draw the line between the need and the want...the useful and the useless ?

Perhaps, there is a simple way to distinguish the technology that one needs from the technology that is unnecessary, superfluous and, in all probability, harmful.

Any technology which makes you smarter while you use it and keeps making you smarter and more creative the more you use it, is a healthy and useful technology for you. The technology that makes you dumb and dependent when you use it, is neither very useful nor very healthy in the long run, irrespective of the convenience that it brings.

Most of the time, we use (rather consume) technology that makes the creator of that technology - and those who control the creators - smarter and more powerful, while making us dependent and constantly distracted (which cannot but cause a steady dumbing down over time).

Technology ...Consumer and Creator

A casual glance at the way people use their computers - one of the most powerful tools of modern technology - can confirm the above statements.

Google-maps may make map-reading, navigation and orientation very easy, but it can also decrease our ability to use our sense of direction, powers of observation and of memory to remember and locate landmarks, judge distances etc.

Typically, when we use google-maps our eyes stay glued to the smart-phone screen (the effect is the same even if we are looking at the road while driving...the app tells us everything), but if we were to go old-style with paper maps we would have to constantly look up from the map to scan the surroundings and ensure that we are at the right spot.

Of course, I am absolutely not suggesting dumping google-maps and returning to paper maps. After all the whole purpose of technology is to make life more convenient for humans so that they don’t have to engage in ceaseless manual labour and mundane tasks and engage in more meaningful pursuits instead.

But if the consequence of such “convenience” is a steady process of dumbing down and engaging in such "meaningful pursuits" like spending hours on social media and the "struggles" of becoming an influencer, then it might actually be healthier and more meaningful to return to a life of heavy manual labour (if one can, that is).

The idea is not to turn ones back on technology (it simply cannot be done), but to be aware of the manner in which the technology is owned, controlled and provided to us so that we can be conscious of its effects on us.

GPS and the Kargil Lesson

Talking about the conveniences of positioning systems, that is exactly the lesson that the Indian army learnt the hard way during the Kargil war of 1999. India was denied access to the Global Positioning System (GPS) by USA exactly when she needed it most in the context of high-altitude mountain warfare. The Kargil war was also a trigger event that led to the development of NavIC (’Navigation with Indian Constellation’...the word ‘navik’ means sailor in Hindi), India’s alternative to the GPS. 

Therefore, the simple learning that the above case provides, is that it is alright to be a user of technology, as long as you also play a role in developing it (or at least understanding how it operates), but it can be downright dangerous if you forever remain merely a consumer of technology.

Still, no matter how dependent google-map may make you, it is still a very useful tool. What arguments could one possibly offer to justify the helpless addictions that are caused by the largely useless social-media platforms ?

I am sure the people who develop these platforms continue to sharpen and develop their skills in programming and problem solving, whereas the users continuously lose the ability to use the computer for the main task it was created to perform – to compute. On top of that, the more they use...the more they generate data for these very same companies.

One cannot wait for society to change to protect oneself from such devastating trends...one simply has to jump off this crazy train oneself.

Linux and Synaptic Connections

For me that jump was in April 2016, when I made the switch from Windows to Linux...and never looked back.

I realised that the users of open-source operating systems and software, inevitably, start transforming into developers with time.

Or as my dear friend Titusz Bugya, who introduced me to Linux and taught me pretty much everything that I know about the proper use of computers, once put it jokingly-

"Linux IS user-friendly !! It is a friend of the User...not of the idiot !"

As the Linux beginner starts overcoming the hesitation and fear of the terminal window and has the first conversations with the computer using the command line, s/he begins to hone that most essential and fundamental skill required for solving a problem, no matter how complex -- the ability to formulate a question.

The clearly formulated question leads to the precisely formulated command and that leads to the desired result.

Here, the Unix philosophy, which is also used in Linux, of using programs that do only one thing and do it very well becomes a great tool. It encourages you to break down complex tasks into component parts - which by themselves may not be as overwhelming - and then deploy the appropriate programs to tackle them one at a time. 

It is not just about approaching and successfully completing a task, but about developing a certain way of thinking and approaching a problem - or as the geo-political expert Andrei Martyanov put it in his brilliant book - "to develop complex synaptic connections which are applicable for everyday life."

Some more hands-on stuff...(OR) how Linux helped Jaga Mission in Odisha

In the previous post I had started discussing about the Linux command line and the incredible flexibility and power it provides to the user. 

Using the command line, and progressing (which happens quite naturally) towards scripting and programming, also halts and reverses the "Smart when you create - Dumb when you consume" process.

The simple fact is that we can't depend on an external IT specialist or a ready-made software for most of the problems that we face regularly in our work.

Only we know the specific problems that we face in our particular work environments - and they may pop up anytime. It is impossible to out-source all such problem situations to an external software consultant.

Similarly, there may be many tasks at work, which could be solved and/or automated through the command line or scripts (a series of commands written down in a file for execution). I have already showed some examples in the previous post. 

In this post let me show another example of a slightly higher order of complexity than the ones I showed earlier.

The implementation of Jaga Mission, the flagship slum improvement project of the Government of Odisha, where I worked as a consultant urban planner, involved the creation of a pretty huge geo-spatial database.

In the first phase of the project, about 2000 slums located in 109 cities and towns of the state were mapped using quadcopter drones. The very high resolution (2.5 cm) imagery was geo-referenced and digitized to create the necessary layers of geo-spatial data layers. 

The following were the major data layers that were prepared for each slum settlement -

a) the high-resolution drone image

b) layer showing the individual slum houses

c) layer showing the slum boundary

d) layer showing cadastral (land ownership/tenancy) data corresponding to the extent of the slum settlement.

e) layer showing the existing land-use of the slum settlement

This led to the creation of a pretty substantial geo-spatial database of about 10,000 map layers. In an earlier blog on operational parameters, I have explained how this database was crucial to fulfilling the goal of Jaga Mission of granting in-situ land rights to slum dwellers. 

The geo-spatial data was particularly useful when encountered with complex situations, such as slums located on certain specific categories of land, where granting in-situ land rights may not be possible. 

When this data was handed over to the Jaga Mission office by the technology consultants, the data-sets were organised in a manner which made quick retrieval and analysis difficult.

The individual layers were stored in a series of folders and sub-folders in a manner as shown in the diagram below -

 

In order to retrieve any layer of a particular type (say, the slum household map) of any slum, one would have to first open the folder of the respective district; then the folder of the respective ULB (Urban Local Body i.e. the city) ; then the folder of the respective slum and then the necessary layer(s).

The file names of the individual layers just mentioned the type (e.g. "hhinf" for the household layer; "rplot" for the revenue plot/cadastral layer etc), without giving any further information suggesting the name of the slum or city.

While this is absolutely fine for manually retrieving the separate layer files and operating on them on a Geographic Information System (GIS) software, this method of data management is incompatible with any attempt at programming, automating or quick retieval.

And when we are dealing with 30 districts; 109 ULBs; 2000 slums; and 10000 data layer, then quick and precise retrieval is essential. Any kind of programming or process automation could also be extremely useful. 

For example, it was decided by the Government that slums located on land belonging to the Railways may need to be re-located to alternate sites. The process could be done by filtering the cadastral layers based on land ownership by the Railways and then selecting the houses which intersect with those parcels from the slum-households layer. 

However, given the manner in which the files were named and organised, this process would have to be done manually on a slum by slum basis. In the absence of an army of GIS technicians (something that the Jaga Mission did not possess), the process was bound to revert to an even more laborious process of municipal staff and revenue officials physically visiting the slums and checking if they were located on railway land.

It was almost as if the elaborate digital database had never been created.

Titusz and I wished to rename and re-organise the data-files in a manner which would enable near instantaneous retrieval and processing. But, of course, even to rename the files (in order to enable scripting), we would need - you guessed it - scripting !....or else how to rename 10000 files stored in separate folders and sub-folders ??

So, we wrote a script which would loop over each of the 30 district folders and recursively go down each sub-folder until it reached the bottom-most level where the data files where stored. 

Every time the script would move down a folder level, it would store the name of the folder as a variable. Once it reached the level of the individual file it would rename it by adding the relevant stored variables as prefixes to the original name of the file. The resultant file name would therefore contain the name of the city, the name of the slum and the type of the data layer (there was no need to add the name of the district to the file name).

The following diagram shows the concept behind the script -

 

Once this process was completed, there was no need to store the files in separate folders and sub-folders. They could be kept in a single folder and files of any combination of city name, slum name and type could be retrieved instantly.

Not only did we have fun trying to create a script that would solve our problem by making use of the names of the very folders in which they were stored (which was precisely the problem that we were trying to solve !), but we also ended up creating a fresh system which drastically reduced the time taken for analysis and decision-making regarding all future tasks.

Effectively, we used the problem to solve the problem.

As a direct consequence, it reduced the burden of manual labour which would have fallen on the shoulders of municipal workers and also reduced the problems faced by slum dwellers due to incorrect decision-making in a process as challenging as relocation.

More on those stories in the forthcoming blogs...

 
 

 

 

With great power comes great...Idiocy !

One of the tragedies of computers in the present times is that they are almost always used for things which they are not supposed to be used for. 

As the name suggests, the primary task of the computer is to compute - to take over the boring, repetitive and labour consuming computing tasks of human beings so that the species can focus on more creative and meaningful pursuits (i.e. human pursuits). 

However, if I were to observe the use of the computer by development sector managers and professionals, it would appear sans doute that the electronic computer was created for the sake of making unnecessarily heavy and pointless graphics loaded power-point presentations. 

Indeed, if one counts the amount of hours (sometimes, all the hours) that young professionals spend adjusting images, animations and texts on their power-point presentations one wonders whether computers have increased office-based manual-labour by orders of magnitude instead of reducing it. 

Of course, a major cause of this, which I have already written about in a previous blog , is the continued dependence on proprietary software despite the steady growth of open-source software and operating systems. 

While proprietary software and operating systems treat computers users primarily as consumers of technology (distracting them with ever flashy and "user-friendly" software products, which are designed to make the consumer feel tech-savvy while simultaneously building a technological dependence akin to substance addiction), open-source software and operating systems encourage users to gradually transform into a free community of developers...liberating them from the addiction of any specific product created by a company (for its own profits of course) and enabling them to create their own products which are suited to their needs.

The super-computers in our pockets

In his brilliant talk titled "You should learn to Program", computer scientist Christian Genco said something, which was quite eye-opening and embarrassing at the same time. 

He showed a photo of the Apollo Guidance computer that was created by NASA engineers to do the complex calculations necessary for the Apollo-11 moon-landing mission. He then flipped out a smart phone from his pocket and said, 

"Your cell phone in your pocket right now, has the computing power to do the calculations of a million Apollo-11 missions simultaneoulsy ! NASA scientists in 1961 would have fallen to their knees and worshipped you for having this kind of technology !"

The Apollo Guidance Computer 

Christian then wrapped up the irony by showing a clip of a little video game and said,

"And what are you using it to do ??"

Here we are, faced with a zillion challenges that are hammering our clueless city governments from all sides....the challenge of affordable land; of urban poverty; of the vulnerability of the informal sector; of climate; of traffic; of housing...(the list goes on and on)...and we use one of the most powerful tools ever created by human beings to play video games, spend hours on social media, make colorful powerpoint presentations etc.

Enter the Command Line...may the Force be with You

Many years ago, when I was in high school, it was screens like the one shown below that made me run away from computers.

 

Little did I know that many years later this grim, dark screen - known as the terminal window - would turn into my biggest ally. I shall tell the story of my introduction to Linux in another blog. But for now, let us talk a bit about this dark screen. The terminal window is the one in which you can type in commands to your computer...when the computer responds, i.e. executes your command, you begin to sense both the true power of the machine and of yourself.

Of course, in the beginning the advantage is not very evident and I wondered why I should not go click-click with the mouse on the graphical user interface (GUI). After all, that is the most familiar and convenient way of interacting with computers running Windows on them. 

But very soon the things you can do with the command line starts overtaking all that you can do with your mouse...and then, suddenly, it goes totally beyond the reach of the mouse as if a spaceship from "Star Trek" just jumped into warp speed and disappeared from sight.

Let's say, I want to create a folder called "data" using the command line. I would type the following at the command prompt and press enter -

mkdir data

On the terminal it would look like this -

 

 

But I can easily do the same by right clicking on the windows file manager and then selecting the option for creating a new folder right ? 

But let us now try to make five folders with names data-1; data-2; data-3 etc. Now it begins to get slightly inconvenient to use the mouse.

But on the command line you would just have to type the following line and press enter -

 

And it's done.

But now....let's make 500 folders !

On the command all that you have to type is the tiny command-

That's it ! When you open your file manager, you see this waiting for you -

You have activated warp drive and left the mouse somewhere far behind in the universe. 

Now, imagine that these folders have been filling in over a long time with all kinds of files (word files; excel files; image files; pdf documents etc) concerning your work.

You would create a new folder called data-pdf and copy just the pdf files into it. It may be that all the 500 folders could contain pdf files or only some of them...you cannot be sure. What would be a mouse based way to do this ? Open each folder, select the pdf files and then copy them into the new folder. 

Or you could type the following two commands -

 

The first command -- mkdir data-pdf -- makes a new folder with the name data-pdf.

The second command -- cp  -r */*.pdf  data-pdf -- uses a command called cp (copy) to recursively (i.e. goes inside each and every folder) check for files ending with a pdf extension and then copy just those files into the newly created folder called data-pdf.

 

Well, well ! Now that is something isn't it ? We asked the computer to perform quite a complex search-n-retrieval task and it did it...in much much shorter time than the blink of an eye. 

And we did this with just two lines of ultra-simple commands.

Imagine what all we could do with a series of such commands.

A series of commands tied together....well, that's a program !!

If you could do so much with these mighty single-line commands (and they are mighty as we shall see)....what all can you do with a bunch of them tied together !!

That's what we will explore here.

(to be continued...)