Posts Tagged ‘programming

The Idea

leave a comment »

The idea is simple and have been done by numerous people before. It is a map based directory for a group of small people. The idea is an extension of the work I did years ago (post link). I built this site waiting for an entire day in the university campus for a (very annoying) friend who kept postponing the meeting. We finally met at the end of the day and by that time I had finished the web app. I had recently learned to use server side coding (using PHP), databases (MySQL) and had figured out that hosting is not rocket science. Coupled with some previous experience with Google maps API and JavaScript. I quickly built this crowd sourced project where everyone enters their Name and City and the map shows all the people, aggregated based on the zoom level. Simple, Clear and nothing fancy.

crowdsourced

Once this was posted to the common forums and people started using it. It became clear that I can do more with it. Suggestions kept coming in and after a point all the suggestions amounted to creating a whole new social network. Though I agreed there were loads of things which could have been improved with the app, I felt the usefulness of this map/visualisation was its simplicity and clarity. The things I felt could be improved were

  • data collection from users. Though it is simple as it is, I feel it can be improved by utilising availability of location data and connecting to other platforms (fb obviously). Also possible account management for users, edit and delete their own data.
  • Search and find people. This was never implemented. It could have been very useful for people to search and find others.
  • Some form of possible communication with other users. When you have a map like this and if you are able to find someone in the map, naturally you try to click on the person to see more details and communicate with them (even if it is a link to their email).

Things I specifically don’t want to do

  • Build a suite of information on the users and nag them to fill the information up. No body cares about someone’s primary school or their fourth work fax number. This will just make the whole thing a directory created with a aim to increase the amount of information rather than usefulness of the information.
  • Build a suite of interactions within the users and app while creating user interface elements for those interactions. This will lead into user being presented with loads of buttons& forms and when these UI elements starts covering the entire screen they will be folded into scores of menus. This just defeats the purpose of the entire thing. If people wanted millions of options to interact with the people they know, they can use Facebook or any social network for that.
  • Anything which is not location based or adds value based on location. Since location and mapping is the central theme of this project.

With all these things in consideration, I spent some time sketching out some preliminary ideas shown below,

I’ll explain them in detail in the following post.

P.S: I still think the idea shown in the sketch is too cluttered and needs significant simplification to be anywhere near neat and useful.

Written by sbmkvp

May 9, 2016 at 5:57 pm

Understanding Javascript Objects and JSON Data.

leave a comment »

The first time I heard of JSON (JavaScript Object Notation) when was trying to get data out of twitter. At that time I was new to javascript and was figuring out a lot of stuff at once so the whole thing was very confusing and incredible hard to grasp. Now when I think back, It would have saved me a lot of time if someone just gave a clear overview of how objects worked in javascript and how the same pattern (jargon alert!) when used to transfer data becomes JSON. This is the reason why I am writing this post. As a disclaimer, I am not a programmer but an urban planner and my understanding of this subject is purely based on my practical experience trying to build things relevant in my field so bear with me if any of this is inaccurate or wrong. Please feel free to point out the mistakes.

To start, let us set up the environment for learning. Since objects in JavaScript and JSON data are abstract concepts there is no way for a normal person to understand these without actually seeing them in action. So it is time to open the console in your browser (preferably chrome) and start typing. The console in chrome can be started by pressing (cmd+alt+j or ctrl+shift+j) and you can just type in the commands one by one. It is as simple as that.

Chrome JS Console

By default the console opens in the global namespace of the tab which is open. In plain english the console is like a virtual space where you create, destroy, modify objects and these objects gets displayed in the browser window (rendering) based on their properties. For example the tab has an object associated with it, the html document, the body and every element (image, text etc) has an object associated with it. This is called DOM – Document Object Model which is used to manipulate HTML elements with JavaScript. The point is this is a virtual space to contain objects and this is where we would be working. This space already has a lot of objects and one shouldn’t confuse them with our own. To check our understanding lets type ‘window’ in the console and press return. It returns some text, this is the object which denotes the chrome window. When you click the triangle in the left side (expand it) you can see all the contents of the object.

Screen Shot 2014-12-03 at 16.01.29

Now typing window.innerHeight or window.innerWidth will give you the height and width of the window. You can resize the window and type it again to see it changing. Here the important things are, 1) We can create objects in this virtual space with properties to represent stuff we see (a window in this case) 2) the “.” denotes the property of an object (i’ll elaborate this later).

Now it is time for us to create our own object. Lets say this object is a digital representation of yourself. Lets start by creating the object by typing me = {}. This literally translates to english as ‘me’ is an empty object. The curly brackets denote that ‘me’ is an object and it is empty since it has nothing in it. Now typing ‘me’ again and pressing enter will return an empty object. Now lets go one step ahead and ask my name. Since it is a property, you use the “.”. So type “me.name” and see the result. It says it is “undefined” because we have created the object but never defined any properties.  Lets set one by typing me.name = ‘bala’ (note the single quotes. i’ll explain later) . Now asking me.name will return the name “bala” and asking for the whole object ‘me’ will now give the properties as well.

Screen Shot 2014-12-03 at 16.16.35

If you have come this far then you understand object, its properties, how to create and return them. Now it is time to understand some data structures (this sounds way too complex than necessary). For practical purposes we just have to know 4 major data structures – Number, String (of characters), Array (of elements) and Boolean. Most of the data we use will fall under these data structure. Numbers are stored as is without any special way to mark them. for example, Typing in me.age = 25 will set the property ‘age’ in me as a number which is 25. Strings are denoted by double/single quotes around them if the quotes are not present then it is considered as a name of an object (variable) and will return an error if it cannot find one. for example, me.city = london will return an error while me.city = ‘london’ will be OK. Arrays are collection of data in a specific order and is denoted by square brackets. for example, me.address = [221,’b’,’Baker Street’] will set the address as an array of 3 elements. typing me.address will return and array, while typing me.address[0] will return 221. the square brackets with a number is the way of accessing an element within an array when you know its place (the counting starts at 0). Try to get the street name out of the ‘me’ object.

Screen Shot 2014-12-03 at 16.48.55

A string is like a special case of array (with only single characters) so you can do the array type of queries to strings as well. for example, me.city[0] will return ‘l’. Boolean is either ‘true’ or ‘false’ (without quotes). for example, you can do me.graduate=true. This helps where you can directly use this instead checking for conditions (like if else statement). With this 4 basic data type you can create a model for almost every kind of objects we usually encounter. To sum up, we create objects with curly brackets, object is something which has properties, properties can be of various types of data (number,string, array, boolean), we set and access properties of objects by using the ‘.’, arrays are created by using square brackets and we set and access contents of array by using square brackets.

There is one more thing we need to know which is that there is a simpler way to create objects than setting properties one by one. which brings us closer to the JSON. instead of using the ‘.’ we can directly write the contents of the object using ‘,’ and ‘:’. So combining all the steps above in creating the me object, we just do,

Screen Shot 2014-12-04 at 18.23.37

With these basics we can now move to interesting stuff – Nesting and References.

Nesting:

When we talked about data structures though we talked about 4, we actually learned 5. The fifth one is objects. This means that a property of an object can be an object. This introduces amazing capabilities to javascript objects. for example, me.education = {} will create an empty object for education. and me.education.school = ‘kvp’ will set the property for me.education. This process can be theoretically repeated forever (if the memory permits).

Screen Shot 2014-12-04 at 17.56.38

This has two major significance. 1) While modelling sparse data, this makes our object memory efficient (this is the reason it is used in data transfer). for example, a table which shows scores of 5 students in 10 different courses and every student attends 3 courses can be modelled as a javascript object as shown below,

Screen Shot 2014-12-04 at 18.13.21

Screen Shot 2014-12-04 at 18.12.52

You essentially don’t need to have null/empty spaces. If something is null then it is just not there. 2) you can model infinitely rich objects by nesting different types of objects together. e.g. in the above example if some course has two markers and two scores you just introduce and object instead of the number, no need to change your structure of the data ( schema ).

Screen Shot 2014-12-04 at 18.28.44

References:

This is another way javascript optimises memory usage. Every object’s association with its name is just a reference. for example create an variable a = 20, create another variable using this variable b = a , check both variables. now do b = 40 and check the value of a. The results are shown below. This is logical and is what you expect to happen.

Screen Shot 2014-12-04 at 18.34.36

Now try and do the same with objects, a = { name:’bala’, age:25 } ; b = a ; b.age = 27. Now check the contents of a!

Screen Shot 2014-12-04 at 18.42.44

This behaviour is because JavaScript does not duplicates objects in memory when assigned to variables but just references the object to the variable name. This allows us to create infinitely rich data structures with finite memory. Funny example is below,

Screen Shot 2014-12-04 at 18.43.53

That concludes our overall introduction to understanding objects in javascript. With this foundation, let us move to JSON.

JSON

To be brief, JSON is just a notation to create javascript objects. It is exactly the same as what we discussed above. The only thing which is new is the property names should be enclosed in quotes (since JSON has no concept of variables). You evaluate a JSON string and it gives you a JavaScript object thats it. Nothing more. While thinking about JSON, as an alternative you can think of it as a collection of key value pairs (this kind of thinking helps while working with php). Keys are always strings and values can be anything (even another collection of key value pairs).

Screen Shot 2014-12-04 at 18.51.11

The difference between JavaScript objects and JSON is that JSON is a notation i.e. it is a text file written outside javascript and cannot be executed. It is a normal data / text file like csv. You can essentially read a csv as a string, split it into parts using the newline and comma characters and create an object/array in javascript but with JSON it is much simpler (just do eval) and more powerful (nesting).

Thats it. Hope this helps absolute beginners as myself in understanding these concepts faster.

P.S. If you have come this far, congratulations! You now know how data is modelled and stored in mongodb! All you have to do to use mongodb is learn commands.

Written by sbmkvp

December 3, 2014 at 5:09 pm

Cellular Automata and Game of Life.

leave a comment »

As a part of the digital visualisation module with the M.Res Programme, we were building cellular automata models with processing. The exercise was really interesting especially the Conway’s game of life where small amount of rules and coding generates a animation which is sometimes rich and endless. To be more specific the model is set in a plain canvas which is divided into equal grids and each cells is either alive/dead. There are three rules

  1. If the number of neighboring cells (n) of a cell is less than the loneliness limit then an alive cell dies
  2. If n is more than the overcrowding limit then an alive cell dies
  3. If n is equal to the creation condition then a dead cell becomes alive and rest stays the same.

I was experimenting with the same model and was trying to create something out of the game of life model by changing some of its rules. The results are as below,

1) Original Game of life:

Here the parameters are, loneliness limit = 2, Overcrowding limit = 3 and the creation condition = 3. This is by far the most interesting set of parameters which creates a life like situation where organic patterns emerge and spread and die. It even results in some mobile ‘organisms’ which can move along the canvas.

.

2) Space Filling Fractal:

Here the parameters are – Loneliness limit = 2, Overcrowding limit = 1 and the creation condition = 5. When the initial condition is a alive cell in the sides of the canvas (not the corners), the alive cell goes on to create a fractal growth which kind of fills up the space.

.

3) Sierpinski’s Gasket:

Here the parameters are – Loneliness limit = 2, Overcrowding limit = 2 and the creation condition = 1. When the initial condition is such that all the cells in the diagonals are alive, it goes on to create a Sierpinski’s gasket in the corners of the canvas.

.

4) Growth and Edges:

Here the parameters are varied through out the simulation and it shows how an object can be created by using a normal initial condition and using a combination of parameters (please note that the parameters are actually changing during the video) which are controlled manually. Even the edges of the object generated can be isolated by just changing parameters.

.

The code which resulted in the above videos can be downloaded here. The instructions are as below

  • press ‘r’ to reset the canvas to blank.
  • press ‘u’ to start or stop the simulation.
  • click and drag in the canvas to introduce alive cells.
  • press ‘a’ to introduce alive cells all  along the mid points, press ‘s’ to introduce alive cells along the diagonals and press ‘x’ to introduce both.
  • press ‘-‘ or ‘+’ to decrease or increase the overcrowding limit and press ‘[‘ or ‘]’ to decrease or increase the creation limit.

Have fun. p.s You need to have processing to run the above code. which can be downloaded here.

Written by sbmkvp

March 28, 2013 at 4:16 pm

Accidents in United Kingdom over 2010 – Animation

leave a comment »

Continuing from the last post, this is a visualization of essentially the same data but with a temporal element as well. The video below shows the occurrence of road accidents in UK during 2010 as it happened in time. Each frame in the video adds an hour of accidents to the canvas as bright blue dots and a second in the video is equivalent to 2 days in real time. The light blue shadow is left behind by previously occurred accidents building a cumulative map. As before, the animation is built with processing + blender and the data is cleaned with R and excel. To be honest there nothing much to observe and infer from the visualization at this level except for the small breathing effect which may be due to the peak hours and nights in traffic and how the accident locations correlates with heavily populated areas with dense roads. The video is embedded below and I would recommend increasing the HD setting to 720p for a good quality, comprehensible video.

The source for the data is from UK Data service at here.

Written by sbmkvp

March 26, 2013 at 5:51 pm

Experiments with Google Maps API

leave a comment »

After spending lot of time with the KML documentation and C, It was time for me to go beyond the associated platform restrictions (C programs are compiled for a single platform) and start doing the computing and displaying maps on the web. This sparked my venture into the Google maps API and  JavaScript.

The starting point of this entire exercise was my short internship with Report Bee (A tech start-up with the vision to revolutionize the education industry).  My job there was to inquire into the ways in which geographic visualization can enrich the analysis of the data present in the results of public examinations in Tamil Nadu and device a frame work for its implementation. It was an amazing ride to for me to start from absolutely zero, with an oversimplified view of things and vague ideas and go on to find out that the whole thing is neither simple nor easy as I thought. Though the exercise proved to have an extremely large scope to be covered in such a short internship, I picked up a lot of knowledge while figuring out my way.

My first challenge, after getting lost in the quagmire of tools and platforms, was to finalize a platform to  learn and work on. The main dilemma was between Google Maps and Open Street Maps. While I liked the freedom and openness of OSM with the base data for the maps, I also found Google Maps really compelling, with its relatively simple JavaScript API and extensive documentation. Though there was a lot of restrictions introduced in Google Maps recently, I chose it over OSM considering its simplicity and my previous experience with KML. Moreover almost all the maps API (Google maps, Map-Box, Bing Maps etc.) had JavaScript in the core so I thought may be it won’t be hard to learn to use others platforms if I get used to one of them.

So after almost three weeks of wrestling with Google Maps API and JavaScript. The end result was that I was able to produce a website with a custom styled map highlighting some of the features in the base map with some kind of user input to point location and has an implementation of Geo-coding API for finding and navigating to addresses. The whole process was extremely interesting and especially JavaScript surprised me with its simplicity and flexibility. I have attached a screenshot of the result below and the website can be viewedhere. (http://balaspa.50webs.com/mapv2/)

Experiments with Google Maps API.

Results

The map highlights all the parks in green colour. The instruction are as below

  1. Zoom and Pan can be done through the mouse as it is done in Google Maps.
  2. Clicking on the map creates a Placemark and adds it to the map.
  3. The ‘Eye’ button on the top right corner toggles the visibility of the Placemarks.
  4. Double clicking the eye clears all the Placemarks.
  5.  The search box below navigates to the location typed on pressing the ‘return’ key.

Since I have covered a lot of basics in this exercise (Customized Base map, simple ways to input and output data, rudimentary ways to build a UI over the map), my next step would be learning to organize a data sets in a server side database and plotting it on the map and also to look into the Open Street Map API parallelly.

Update (21 August 2013): Was randomly reading about the Google speech recognition API for chrome and stumbled upon a simple way to integrate speech based input fields in websites. Updated the search box with speech recognition. In my trials it looks clean enough and works well enough. Wish there were more options in controlling the behaviour of the box but I think its just a matter of figuring out.

Update (21 August 2013): Updated the speech recognition again. Now press “ctrl+shift+.” and speak the place name the page directly takes you there. No more clicking and pressing enter.

Written by sbmkvp

October 29, 2012 at 4:21 am

Programming Part II

with one comment

This is a follow-up to the previous post Programming a new start… After familiarising myself with the old friend – C and completing a simple program, It was time for a bigger, better and more complex stuff. The search was around a problem which involves geo coordinates and their manipulation into producing a visual output.  There was 2 parts to the problem, one was the analysis part and the other was the visualisation part. C was enough for the analysis part, with its insanely simple tools; one can do anything to a pair of coordinates given to it. It was the visualisation part which was proving to be  tricky, provided the complexity of the system which needs to be visualised (since it is not a simple Cartesian plane which needs to be mapped) and the graphical operations which needs to be carried out, which I felt would be really difficult do in a platform like C. I kept on searching for something in Java, JavaScript, etc. where I can draw things visually on a screen with coordinate data and save it in a file structure which can be read by others. The search went on for a long time until I found KML.

Keyhole Mark-up Language (KML) proved to be the solution to all the geographic visualisations I needed to do. The advantages were that it had XML roots – which makes it easy to learn, It had simple vertex based mapping – where all the drawing and projection systems are taken care by the viewer (Google Earth) and I was a simple text based file system – which enables one to create a KML file by just writing in plain text. By combining, KML and C one gets the power to carry out complex calculations and at the same time, store them and display them through a standard format which is widely used. So now my aim was to create a program which creates a KML file from a set of data provided to it in a standard format.

After two days of continuous coding and debugging, the final version of the program is here. What it does is to take input from a text file as pointed by the user and looks for points defined by delimited data (Space, tab or newline) in four columned arrays [name, latitude, longitude and distance] and draws a circle around the points with a radius of distance specified and stores those polygons in a KML file which can be viewed in Google earth. Since I don’t want to get into really minute details of the concepts and directions, as I did last time, I will mention the broad concepts which I learned and extensively used in the program below,

  1. File management in C – Opening, reading and writing text files through C.
  2. KML Specification – Constructing different types of geometry in Google Earth through KML files
  3. Geometry and projection system – Ellipses, Circles, spheres and how they are defined in Cartesian geometry and how do they transform when projected on earth.

The program as of now has two major drawbacks,

  1. It assumes the earth to be sphere and not a spheroid which can cause small errors in extreme latitudes
  2. There cannot be any space in the name field of the data, which will cause the program to read it as a delimiter and go berserk with the rest of the data.

The download link is given below. Feel free to point out any blunders, problems and ways for improving the program.

Download Link:

https://sourceforge.net/projects/kmlbuffer/files/KmlBuffer/

Screenshots:

Input

Processing

Output

Written by sbmkvp

May 24, 2012 at 12:56 pm