In the first part of this blog we had discussed how the various steps of a particular planning problem (in this case the slum-proofing problem of Jaga Mission) can be articulated, algorithmised and then automated. In the second part we shall see how the specific commands in the computer program work.
The technical steps necessary for achieving the goal (given the capabilities and constraints of government organisations executing the mission) were identified and are listed below -
(a) Identify the location of existing slums
(b) identify vacant government land parcels near the existing slums
(c) check them for suitability
(d) generate map outputs for further visual analysis and verification.
It is clear that the problem solution involves spatial analysis tasks to be performed on a Geographic Information System (GIS) software.
While QGIS is the more familiar and user-friendly option, my preferred software for tasks like this is GRASS, which stands for Geographic Resources Analysis Support System.
GRASS is an extremely powerful and versatile open-source software, which was originally developed by the US Army - Construction Engineering Research Laboratory (CERL) and then by OSGEO - The Open Source Geospatial Foundation.
A very useful feature of GRASS is that when you undertake any operation on it (e.g. clipping features in one map layer using features in another map layer), the command line version of the operation is also shown.
This is because GRASS is not just a point-and-click software (software which rely on a Graphical User Interface - GUI - for performing the operations by clicking with a mouse), but it also contains a command line option.
Most computer users are unfamiliar with the Command Line Interface (CLI) - even intimidated by it (I certainly used to be not so long ago). They, therefore continue to work on the user-friendly environment of the GUI and doing all their tasks with the mouse. However, by doing that they fail to harness even a fraction of the power of these wonderful machines - there is a reason why users who work comfortably with the CLI are called Power-Users. I have written about the advantages of the CLI in earlier blogs.
When GRASS is launched, it opens three windows simultaneously - the Graphic console (for performing operations using the mouse) ; a display window (for viewing the maps and results of operations) ; and a command line window (for performing tasks by typing in commands).
An interesting thing happens when you do any GIS operation on GRASS. Let's say you wish to execute the module called v.extract , which creates a new vector map by selecting features from an existing map (quite like exporting selected features to a new map layer in QGIS). The series of steps is similar to that in QGIS, where you specify the input file name, the output file name (of the new file), any queries that specify which features are to be selected etc.
You will notice, that while you are specifying these details by clicking with the mouse and entering file names, the following line is getting typed at the bottom of the module window -
This line is basically the command line version of the v.extract operation. This is how it reads --
< v.extract input=bbsr_slum_houses@training where="ward_id = 16" output=ward_16_slum_houses --overwrite>
The command begins with the name of the module, then specifies name of input file, name of input file and the sql query (in this operation the query selects all slum houses that are located in ward 16). The optional --overwrite can be added if you run the operation multiple times and would want the previous output files to be overwritten automatically (useful during scripting/programming).
When you click the "copy" button in the module window, this command line gets copied to memory. You can then paste it in a text file to make it a part of your program.
A good practice is to run all the operations manually once using the mouse, just so that you are clear about all the operations and all the command line versions of the operations. With practice you would not need to copy the commands. You will simply be able to type them out from memory or my referring to the manual page of the respective module.
Turning individual commands to a script
Commands are very powerful because instead of clicking with a mouse many times, you can just type a short line and get the job done. But the main power of the commands is that you can write a bunch of them down in a sequence in a text file and execute it as a program to automate all your computing tasks.
Consider the following 4 commands written down in a sequence -->
(1) v.buffer input=centroids output=buffer type=point distance=$buffer
(2) v.clip input=orsac_tenable clip=buffer output=vacant
(3) v.out.ogr --overwrite input=vacant output=$LAYER_OUTPUT/$ulb\_vacant_$buffer\_m_buffer.gpkg format=GPKG
(4) db.out.ogr --overwrite input=vacant output=$LAYER_OUTPUT/$ulb\_vacant_$buffer\_m_buffer.csv format=CSV
Of course, it looks a bit overwhelming ! But nothing to worry, cos everything useful feels a bit overwhelming at the start. Here is what is going on -
In step 1, the v.buffer module draws a buffer of user defined distance around the centroids of slums and saves it in a file called buffer. In step 2, the v.clip module clips features from the land parcels layer using the buffer layer file. In step 3, the v.out.ogr module exports the clipped land parcels as a gpkg vector file and stores it in your chosen folder. In step 4, the db.out.ogr exports the attribute table of the clipped file (containing details of the land parcels) as a comma separated value (csv) spreadsheet file and stores it in your chosen folder.
When you run these commands as a script then all these tasks get performed automatically in this sequence.
It is clear from the above, that you can tie together any number of commands in an appropriate sequence to handle analytical tasks of varying levels of volume and complexity.
GRASS + BASH
Bash - an acronym for Bourne Again Shell - is basically a "shell", a program in Unix like operating systems such as Linux, which is used to communicate with the computer i.e. it takes keyboard commands and then passes them onto the operating system for execution.
The unique thing about Bash is that it is not just a powerful method for passing commands to the operating system but also an effective programming language.
The advantage of combining GRASS with Bash is that you can take the GIS commands of GRASS and use them as it is in a Bash script without any change in syntax by adding a flag called --exec at the start of the command.
By doing that you can not only execute the GIS tasks but combine those tasks with all the other programs and the full power of your Linux computer.
For example, in the same script, you can select GIS files of a few specific cities, undertake all the operations that your need to undertake, then save them in a separate folder, extract the attribute data as csv files, undertake data analysis tasks on them, put them in some other folder, turn both the spreadsheets and the output maps into pdf files etc etc.
I guess this is already somewhat of an overload...and I feel tired of typing too ! I will show the combination with Bash in the next blog.
The request and the reward
Using the command line and scripting with GRASS and Bash are extremely powerful, flexible and fun processes, but they do request a readiness to learn and an openness to be creative.
The rewards...well they are immeasurable.
No comments:
Post a Comment