Basic R revision - Part 3

Dataframes : Datasets in R

When you work with (extremely) large datasets and data frames, your first task as a data analyst is to develop a clear understanding of its structure and main elements. Therefore, it is often useful to show only a small part of the entire dataset. To broadly view the structure of a dataset use head() to look at the header columns and first few observations. (tail() shows the last few). Another method is to use str() which gives you the number of observations (rows), number of features or columns for each observation, a list of variable or column names and their datatypes, with their first few observations. Other useful functions are names() which gives column names, and dim() which gives a vector of two elements - nrows and ncols of the dataframe.

Creating a data frame

Normally you need your data in a very customized form before you can run any statistical algorithms on them. You can either perform that customization at the database level, that is, by querying in SQL to generate your output of the most suitably customized form, or, you can import the raw data onto your R (or Python) environment as it is, and use R (or Python) to create custom dataframes afterwards. (I do not currently have an opinion on what is the best practice - mostly common sense dictates what to do - but I will add to this post if and when I do have any nuggets of wisdom on this).* Here we will learn how to use R to create custom dataframes.

added 21Oct2015
I now feel that it is generally better to do the latter - that is - don't try to work with the SQL query too much to get customized data output - there are much better tools to deal with customization at the language level. R has data.tables and dplyr, for example. For an example, suppose there are two cols a and b and you only want to output the part of the whole dataset where a>5. Easily do-able in SQL. But suppose you only want to output the part of the whole dataset where a+b>5 - not doable as far as I know in SQL. But at R level you can do it.

We can use the data.frame() function to wrap around all the vectors we want to combine in the dataframe. All the vectors, of course, should have the same length (equal number of observations). You can think of this function as similar to cbind, except it deals with vectors of potentially different datatypes. It's not really that similar to cbind actually, as each argument to to data.frame() should be a vector, whereas arguments to cbind can be some vectors and some matrices too!

planets     = c("Mercury","Venus","Earth","Mars","Jupiter","Saturn","Uranus","Neptune");

type        = c("Terrestrial planet","Terrestrial planet","Terrestrial planet","Terrestrial planet","Gas giant","Gas giant","Gas giant","Gas giant")

diameter    = c(0.382,0.949,1,0.532,11.209,9.449,4.007,3.883);

rotation    = c(58.64,-243.02,1,1.03,0.41,0.43,-0.72,0.67);

rings       = c(FALSE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE);

# Create the data frame:

planets_df  = data.frame(planets,type,diameter,rotation,rings)

Indexing and subsetting in dataframes works similar to matrices.

To get diameters of the first 3 planets in planets_df, we can use any of the following:

fpd1 = planets_df[1:3,"diameter"]

fpd2 = planets_df[1:3,3]

fpd3 = planets_df$diameter[1:3]

Example to get only those observations of dataset where planet has rings: planets_df[planets_df$rings,]

For an alternate way to do the same thing, use subset(): subset(planets_df, subset=(planets_df$rings == TRUE))

Use this way to get observations of dataset where planets smaller than earth: subset(planets_df, subset=(planets_df$diameter<1 span="">

To add a new feature or column or attribute to the dataframe planet_df, let's say sun_closeness_rank, simply define it while referring to it as an attribute of that dataframe:

planets_df$sun_closeness_rank = c(1,2,3,4,5,6,7,8)

Sorting a vector in R

The order() function, when applied to a vector, returns a vector with the rank of each element.

For example, order(c(6,3,8)) = {2, 1, 3} vector. Now this vector can be given as index to the original vector, to get a sorted version of original vector.

a = c(100, 9, 101)

order(a)

[1] 2 1 3
a[order(a)]

[1]   9 100 101

Sorting a dataframe by a particular column

For example, if we want to sort planets_df by diameter descending and create largest_first_df:

positions = order(-planets_df$diameter)

largest_first_df =planets_df[positions , ]

Being regular

1. Regularity is everything. Learning something for 2 hours for 25 alternate days is far, far superior to learning the same thing for 10 hours each on the first 5 days and then not coming back to it. On day 51, you will be in a much better position by following the first strategy.

2. If you don't intend to keep using a skill (WebDev, ML - anything) at least 2 to 3 times a week, for at least a couple of years, you might as well not learn it. You will unlearn it in as little as two to three months, and if you need that skill again you will have to start from the very beginning, making you question why you spent all that time learning it in the first place.

3. Set all non-focus but essential things on a reminder until it becomes an autopilot thing. This includes things such as exercising - essential, yes, but should not occupy your mental space and time. The time you spend actually exercising is all the time you should devote to it, nothing more. 

Basic R revision - Part 2

Factors

Factors are used to store categorical variables, where categorical variables are those whose value can only be one amongst a well-defined, discrete set of values. For example factor_gender is a factor that stores variables that can contain elements: "male" and "female".

To construct a factor variable out of a vector of values, just wrap the vector using factor(). For example:

> gender_vector = c("Male", "Female", "Female", "Male", "Male")
> factor_gender_vector = factor(gender_vector)
> factor_gender_vector
[1] Male   Female Female Male   Male 
Levels: Female Male

Categorical variables are of two types: nominal and ordinal.
factor_gender would be nominal as there is no grading from lower to higher between male and female unless you are a sexist asshole.
factor_bondratings would be ordinal as there is a natural grading, where we know :

AAA > AA > A > BBB > BB > CCC > CC > C > D

In R, the assumption in for the categorical nominal variable to be nominal. If you wish to specify ordinal, use the order and levels keywords:

temperature_vector = c("High","Low","High","Low","Medium")
factor_temperature_vector = factor(temperature_vector, order=TRUE, levels=c("Low","Medium","High"))
> factor_temperature_vector
[1] High   Low    High   Low    Medium
Levels: Low < Medium < High

Renaming the elements of a factor variable

Use the levels() function to do this.

> survey_vector = c("M", "F", "F", "M", "M")
> factor_survey_vector = factor(survey_vector)
> factor_survey_vector
[1] M F F M M
Levels: F M

> levels(factor_survey_vector) = c("Female", "Male")
> factor_survey_vector
[1] Male   Female Female Male   Male 
Levels: Female Male

Note that it is important to follow the correct order while naming. Using
levels(factor_survey_vector) = c("Female", "Male")
would have been incorrect, since I had run the code earlier to see the unnamed output being "Levels: F M"

Using summary()

summary() is a general R function but it's very useful with factors. For example:

> summary(factor_survey_vector)
Female   Male
     2      3

If a factor is nominal, then the comparison operator > becomes invalid. See the following (continuation) code for my favorite proof for the equality of sexes:

> # Battle of the sexes:
> # Male
> factor_survey_vector[1]
[1] Male
Levels: Female Male
> # Female
> factor_survey_vector[2]
[1] Female
Levels: Female Male
> # Male larger than female?
> factor_survey_vector[1] > factor_survey_vector[2]
'>' not meaningful for factors

Comparison operators meaningful for ordinal categorical variables. See:

> speed_vector = c("Fast", "Slow", "Slow", "Fast", "Ultra-fast")
> # Add your code below
> factor_speed_vector = factor(speed_vector, order = TRUE, levels = c("Slow", "Fast", "Ultra-fast"))
> # Print
> factor_speed_vector
[1] Fast       Slow       Slow       Fast       Ultra-fast
Levels: Slow < Fast < Ultra-fast
> # R prints automagically in the right order
> summary(factor_speed_vector)
      Slow       Fast Ultra-fast
         2          2          1

> compare_them = factor_speed_vector[2] > factor_speed_vector[5]
> # Is data analyst 2 faster than data analyst 5?
> compare_them
[1] FALSE

So Analyst 2 is not faster than Analyst 5.

Basic R revision - Part 1

A random useful function

To get the data type of of variable in R, use the function class().

my_numeric = 42
my_character = "forty-two"
my_logical = FALSE

> class(my_numeric)
[1] "numeric"
> class(my_character)
[1] "character"
> class(my_logical)
[1] "logical"

Always remember: Python/C++ vector indices start with 0, R vector indices start with 1
 Subset a vector in R, use vectorname[c(starting index: ending index)]
If disparate (non-adjacent elements): vectorname[c(index1, index2, index3 ..)]

Compare to Python:
Subset a vector in Python, use vectorname[starting index: ending index + 1]
Note that index numbers will be defined as per Python convention
Suppose from a vector v = ['P','O','K','E','R'], we need to output ['O','K','E']
In Python, use v[1:4]
In R, use v[c(2:4)] or just v[2:4]

Also to get all elements in Python a way is to do Mymatrix[3, : ] (gets row 3)
To do the same exercise in R the way to do is Mymatrix[3,  ]

Comparison Operators in R vs VBA and Python

Comparison Operators in R and C++
<, >, >=, ==, !=

Comparison operators in VBA
<, >, <=, =, <>

Python supports both != and

<>

For equality, Python supports == (like R and C++)

In R you can use comparison operator between a vector and a number and get a binary vector which compares each element of the vector to that number.
(Not sure if you can do that in Python. Will check later and update.) Also, you can use that binary vector as an index to get a subset of the original vector.

Matrix in R: To construct a matrix in R you need to add a matrix() wrapper to a vector. e.g. matrix(c(1:9), byrow=TRUE, nrow=3)

Naming elements of a vector and rows/cols of a matrix

Naming can often be useful later. Syntax is simple:
For vector:

vectorv = c(2,3,4)
names(vectorv)=c("a","b","c")
Now, vectorv[“a”]=2

Now, vectorv[“a”]=2

For matrix:

new_hope = c( 460.998007, 314.4)
empire_strikes = c(290.475067, 247.900000)
return_jedi = c(309.306177,165.8)
# Construct the matrix
star_wars_matrix = matrix(c(new_hope,empire_strikes,return_jedi), nrow=3, byrow=TRUE)
# Add your code here such that rows and columns of star_wars_matrix have a name!
rownames(star_wars_matrix) = c("A New Hope", "The Empire Strikes Back", "Return of the Jedi")
colnames(star_wars_matrix)= c("US", "non-US")

Another way can be to include these in the matrix definition itself:

movie_names = c("A New Hope","The Empire Strikes Back","Return of the Jedi")
col_titles = c("US","non-US")
star_wars_matrix = matrix(box_office_all, nrow=3, byrow=TRUE, dimnames=list(movie_names,col_titles))

Summing all elements of entire rows or columns, or summing all elements of any vector

To do row sums or column sums in R for a matrix just use rowSums(matrixname) or colSums(matrixname). Note that it is important to capitalize S in rowSums or colSums. Another way can be to reference the needed vector by using something like Mymatrix[3, ] and then wrapping sum() around it.

Combining/Appending functions

cbind(vectorname) can append a vector to an existing matrix as a new column, provided vector's length is same as number of matrix rows. Similarly, rbind. Note the similarity to c() wrapper to construct any vector.

Arithmetic Operators

+,-,*,/ work in an elementwise way for both vectors and matrices
matrix1 * matrix2 does elementwise multiplication, not matrix multiplication as in Linear Algebra for which we use %*% in R

Carbs vs Fats

In a comparison of macronutrients, the often misunderstood fats are way better than carbohydrates. Fats, especially the unsaturated ones, provide several essential functions such as protecting our inner organs, maintaining good cholesterol levels and reducing bad cholesterol. Outside of them, Omega 3 fats found in foods such as Tuna, Walnuts and Beans are one of the healthiest things you can consume for your brain function - improving memory, fighting depression, bipolar disorder and ADHD. In addition, it is good for your cardiovascular system and bone joints. Even some saturated fats, such as those found in Desi Ghee, help break down other hard to digest food and reduce the negative effects of other fried and spicy food you eat, making them easily digestible. The one category of fat that is unequivocally unhealthy is trans fats, which would be all fatty food items with a good shelf life - biscuits, chips, pies, donuts, cake etc. In comparison, there are no “essential” carbohydrates, that is, there are no essential body functions that require carbohydrates. So the only function carbs serve is to provide energy, which is provided in just as much quantity by more functional nutrients like fats and proteins.

Indian diet is very heavy on carbs, which provide energy but aid no body function, except that a basic amount is needed to aid digestion. You would be surprised that that basic amount is so little as 3 slices of bread a day for a full grown adult, and that is if that were all the carbs he were consuming. Of course, an average adult will also be consuming carbs in decent quantities from vegetables, beans and fruits.

__

Disclaimer: This is a log for my personal use and does not constitute medical advice from me. I am not qualified to give medical advice to others and you should consult your doctor before making any nutritional or medicinal changes. 

Making good on promises past.

So on the first day of the year I promised to write a blog post here about something I learn "almost daily" and then stopped after day 2. That is a familiar theme with new year resolutions. I see a problem there in that statement I made. It was the qualifier "almost". Qualified promises are like qualified love: imaginary. Secondly, vague goals detract from implementation. So, yeah, I shamelessly claim that I'll be updating it daily. Yes, that would be every day. One of the main motivations for writing things down on the blog is the belief that writing notes is not only helpful as a revision tool for committing things to memory, but also that there are things you learn while writing about a subject that you had not learned while reading about it before.

I have been spending a few hours everyday learning some interesting stuff, only I never got around to writing about that here - so I'm hoping to also backfill some entries retroactively for the lost 26 days of the year whenever I go back to revising that stuff.

Here we go again.

World changing developments

I read a lot of history stuff at the beginning of this year, where my focus was on breadth rather than depth. I did not study in great detail about any one period, wanting instead to get some sort of a hang of everything. Of course, I failed miserably. But I did come out at the end of those five or six days subconsciously wondering about the developments, from a very wide angle camera, that I thought really changed the world the most. Even though a lot of entries on my list are so obvious when I look at the list ex-post that I wonder why I needed to spend those days to arrive at it, but, in any case, here's my list:

1) Discovery of fire - Because duh uh.

2) Discovery that seeds can be cultivated - This was the single most important developments that changed humans from hunting gathering nomads to co-existing people in civilizations.

3) Mass Production and standardized parts - Pioneered by the development of the Chinese crossbow

4) Concrete - Changed the way structures were built forever. Introduced by the Roman emperor Claudius while getting Aqua Claudia built, an aqueduct colossal even by modern standards.

5) Printing Press - Because it changed literacy from 2-3% to 70-80% in the western world in a matter of decades. That is a great leap for mankind.

6) Steam Engine - Because it paved the way for the Industrial revolution which would give us trains, cars, other engines, heavy machinery, and pretty much everything.

6) Malaria and Polio vaccines - Because you can now expect to celebrate your kindergarten birthday party.

7) Internet - Because I say so.

And now, some trivia that I found fascinating. Two of the greatest empires - the Egyptian and the Roman - were decimated by people one would never guess were capable of it. The Egyptian one, the largest empire of its time and perhaps the most long lasting one ever with a period of continuous rule between 1000 to 1500 years, was brought down by people nobody knew at that time even existed, and historians still don't know where they came from. They were people whom the Egyptians called "People of the Sea", because they came from the sea. The Roman empire, the biggest  ever, was brought down, once again, by a small tribe of germanic people called The Vandals under the leadership of Gaiseric - not by another big empire of that time such as the Persian or the Chinese.

Finally, next on my history reading is "Guns, Germs and Steel", again a book that focuses on breadth (history of the world from pre-civilization to today) rather than depth into a particular place or era, although it does supposedly go into great depths vis-a-vis outlining some specific themes that pervade all of history. I have heard good things about the book, but I cannot say when I will get around to it. 

Filling time

It's 9:23 PM on a Friday night. I'm alone in the office. If I stand up and look across the hall, I see two hundred computer screens, most of them dead black, some of them blinking with screensavers. "We innovate" lights up the screen in bright yellow, followed by a bold screen-size "passion for performance" in green. A few TV screens hanging from the walls, muted of course, show a nameless man opening and closing his mouth in a magnificently determined way, and the old guy with the crew cut sits before him nodding sagely. The recycled fiber glass on my desk was filled with soy milk an hour ago. I drank it with corn chips from the vending machine. Later, I felt unsatisfied and went back and got a pack of cheese crackers. Then I thought I'd go back and walked out into the freezing parking lot. Suddenly dawned on my it was -12 degree celsius and I had forgotten my ear muffs on my desk. Who'll go back, I thought, and ran to my car about fifty meters away. The car's battery had broken down, so I had to come back anyway. I called car services, and the woman asked me my address thrice. I do not understand how my awful accent could be so awful that each time she mistook my three for a two. You know, it makes no sense to me. The second time, I had made sure to speak three with great emphasis on the ree, and also followed it by saying, you know, like free, like a tree? She repeated the address afterwards, again having noted down two in place of three, and told me she's sorry about my accent. I was, like, I love India. I'm now waiting for the guy to come give my car a jump start.

I never asked for roadside services in India, wasn't even aware that there was some such thing. Three times in my driving life in India, I had to change the tyre. I knew how to do it, and on top of that, each of those times I had friends with me. The first time was late 2009 in Gurgaon, when I used to live in Vipin's house, when Vipin and his dad helped me. Actually, that was the time I was taught how to do it by his dad. Then in the monsoons of 2010, at 2 AM in the night while leaving work, I found my tyre punctured. Vibhor and I fixed it, it took almost forty minutes for us newbies. The third time was a couple of months before I left India, in 2012. I had Kavish, Vaibhav and Gyanesh with me, and this time was the hardest, for none of them helped any more than cracking jokes in the background while I attached the stepney. I do not know why it is a fond memory. Really, I have no idea.

The guy just called. He said he'll be here in five minutes, so I better wind up the post quickly. Actually, it's no admissions essay, so I do not have to worry about an appropriate closing, I can end pretty much anywhere. What'chya gonna do about it?

The fine difference between GDP deflator and CPI

GDP Deflator and CPI or Consumer Price Index are both price level indices, with two crucial differences in the way they are calculated:

1) GDP deflator keeps quantities fixed at the "current year", whereas CPI keeps quantities fixed at the "base year".

So, going from 2013 (assuming that is the base year) to 2014 (current year), the value of the deflator in 2014 will be calculated using weights for different goods that correspond to their weights as observed in the 2014 economy. 

Deflator as of 2014 = Σ [P2014 (i) * X2014 (i)] / Σ [P2013 (i) * X2014 (i)] 

Or in other words, Deflator= (Nominal GDP in 2014 / Real GDP in 2014)

On the other hand CPI keeps quantities fixed at the "base year", so in this case:

CPI as  of 2014 = Σ [P2014 (i) * X2013 (i)] / Σ [P2013 (i) * X2013 (i)] 

(Fairly obvious, but let's note that P's represent prices, X's represent quantities)

2) In both cases above, I used a summation with variable i. It is important to note that in the case of deflator, the "i" loops over all the goods and services produced in the economy of that country, whereas in the case of CPI it loops over a well defined basket of goods that are identified as consumer goods.

This distinction is important for quite a few reasons. Inflation can be calculated both as the rate of change of deflator or as the rate of change of CPI, but the purpose for which that inflation estimate is required determines which of the two methods to use. For example, policy decisions often take CPI into account as it is a better indicator of inflation as it is felt by the general population. 

Secondly, CPI is much more volatile than Deflator. You would expect that to be the case since statistically the Deflator calculation seems to average (weighted-average, to be precise) a whole lot more quantities, and you'd expect the law of large numbers to kick in. But there's also an intuitive explanation on top of that: CPI has a large percentage presence of some of the most volatile sectors, such as, housing, energy and food.

For this reason there is another important measure, commonly called "core inflation", which when considering the rate of change of CPI excludes its volatile components like energy and food which vary a lot merely as a result of short term vagaries of weather, yields etc while not representing any structural change in economic trends.

A word of caveat. Both these measures are hard to estimate with precision, the GDP deflator more so because it has so many more moving parts. CPI estimations, although relatively simpler, have long been tainted with accusations of manipulation by government bodies that estimate them, and often have agendas that outweigh the pursuit of precision. Recently, the startup premise dot com has employed crowdsourcing principles to estimate food inflation from the ground up by having people send prices to them of food they buy everyday while their engine does the almost real time number crunching. You would be surprised at how much their estimates of food inflation differ from the "official" figures, especially for developing, populous countries like Brazil and India. 

Turn of the year musings

I used to think until fairly recently that people don't change. I still think so, by and large, but have mellowed down the firmness with which I think so. On the one hand, a person's character is something I'm still positive does not change, but on the other hand, I do agree now that it is possible for people to start experiencing the same old things in new ways that they had not known earlier. Somewhere in the beginning of last year, I found myself having entered a new kind of life where I spend a massive proportion of my waking hours entirely alone. I wouldn't say it changed the person I am, but things have started to happen that never did before. For example, I get unusually pensive while taking a shower. It works like magic, as if the knob of the shower were a switch into my sometimes dark, often colourful past. The moment I stand before the warm water springing onto me, I have recollections, very fine and vivid and almost hyper-real, of episodes of my past. Only I don't get to choose the episodes. It could be anything from a long day spent waiting for a phonecall, an evening where my dad corrected my spellings, to fighting over the sole plate of chowmein with five of my contending classmates, or reliving the heated anticipation of walking into a fresh acquaintance's apartment while she unnecessarily brings her cat for both of us to play with. It's amazing what a hot shower can do. And how life comes back to the forever nondescript present, the second the shower is stopped.

It's not a resolution, but I've planned to update the blog daily with a new thing I learn. Mostly, it's just to impel myself to keep learning, for I'm starting to realize that time spent away from any kind of learning really rusts your brains and makes you slower. And older. Which reminds me, I don't like getting older. In two months, I will be 29. I'm already at the age that if I were a world-class tennis player, which I'm far from, I would be considered a fading veteran by now. On my 29th birthday, I will take a picture of myself, and compare it to my older pictures, and try to conclude that I hardly look older. I am willing to resort to Adobe Photoshop to reach my goal.

Swimming in a Fish Bowl

This morning, Albert Wallace of the bent back but aristocratic nose woke up disheveled, shortly before a subdued autumn sunrise in a dimly-lit, painstakingly unadorned apartment, and after only a few moments of lying there in a trance, slipped his feet into his fading crimson rubber slippers, and promised himself he is so not gonna take shit from anyone no more. He straightened himself up and took brisk steps to the mirror, because of course the best way to avoid his fears was to confront them, and splashed freezing cold water into his face, furiously, fifteen times. 

“I’m gonna go out today, and I’m gonna talk to people!” he told his mirror image with an unsure enthusiasm, but with such loudness that one might as well imagine he was planning on conquering the Everest in his boxers. Once a popular creature with enviable looks, a reputation for expertise in all things from Probability to Literary Theory, and, even, a guilty taste for the rowdy humor of large groups, he had somehow found it difficult to get back on his feet after his former girlfriend, Anita, had suddenly left him for her long-time best-friend Patrick Boehner, who even Albert had grown a social fondness for, over time. But that was two years ago, and for what it’s worth, Patrick Boehner had by now married another fine woman, and Mr. and Mrs. Boehner are expecting a daughter next month, as some harmless facebook stalking has revealed.

For some months now he had felt no pleasure in most things people seemed to love, and often wondered how he, himself, had shown such keenness for the same activities in the years before. Every time he would discover himself tagged in a facebook photo by one of his acquaintances, the accompanying taglines that were usually on the lines of “Awesomeness”, “Amazing fun” or “Best day ever” left him nonplussed. If they hadn’t mentioned it I would never have known, he would muse. 

In the earlier days, he used to occasionally do stand-up at a local comedy club, and although he still appreciated the mechanics of humor and continued to be able to construct, methodically, laughter on other people's faces, he had found it increasingly difficult to elicit his own laughter, and felt uninspired, even, by the virtue in giving other people a good time. 

He had gradually become unmotivated to excel in his career as an Accountant, and the final straw came two months ago with his rather public firing, since which time, he hasn't applied for another job. It was the day after the firing that he first attended our yoga class. 

But today was different. He looked palpably determined in his freshly ironed, crisp white shirt and linen trousers getting ready for the Interstellar show at AMC New Brunswick. Unusually chirpy all morning, he played his favorite music from The Beatles to The Doors at full volume while he vacuum cleaned the whole place. But it was really when he began humming along and grooving rhythmically to the high points of these happy songs that I really knew today’s different. Very uncharacteristically, he also nudged me to wear my dark azure top, telling me how it enhances my breasts in a “menacing way". I was, like, woah!

I’ve been living with him for the last two weeks, but contrary to what others in the class think, we are not sleeping together; the only reason I’m living here is that we were both scared of living alone. Scared of ourselves, perhaps. We keep each other sane. And useful.  

“Let’s get going now” I cried at 5 minutes to noon, “we will have people from the class, all waiting. We’re also grabbing lunch before, alright!” 
“Just a moment, honey. I am shaving.”
“C’mon now. It’s not a party! And what’s with ‘honey’? I am not your girlfriend, okay?” I said, and put on Breaking Bad on Netflix. I was in the middle of season 5, and it’s getting crazier every episode. It’s so addictive, oh my god. It wasn’t until the episode was over that I realized Albert is still inside. What a bride-to-be, this guy.

“Albert! Dude. I’m gonna go by myself.”

No voice responded.

-- 

The doctors just told me there is scope for revival, but added that it was “in everyone’s best interest to be prepared for all eventualities.”

I don’t know whether we were lovers or just close friends or just two random depressed people who, as a matter of mere convenience, were each other’s support group. Whatever we were we weren’t anymore.

Sid Shoulda Said - Update

Just thought I'd post an update regarding the fictional series "Sid Shoulda Said". I've started posting newer installments of it on another blog, while this one continues to be more or less as 'twas before, that is, more personal and more lethargic.

Should you be interested, I'd be glad to have you follow "Sid Shoulda Said" at ohsid.quora.com. I think following it and commenting on its posts might require a quora account, but rss feeds and bookmarking etc should work just like any other blog and be relatively undemanding.

Sid Shoulda Said - Part 2 - Kruger Ganley, Then and Now

I’m feeling positive about an impending promotion. Last Friday, my boss asked me out to lunch. Well, to go together and pick up lunch from Sarvana Bhawan, to be precise, but that’s almost as good, definitely a signal. Then he got an important call so I had to go pick it up for us both. But why must I worry about a free lunch gone expensive, since what mattered was the signal, and that, as you’d agree, remains intact. I know that because he also complimented me yesterday on my work migrating the trading books to the new platform. “That was helpful”, he’d said. I’m actually a statistician working with market and macro data, so this was not, in the strict sense, a part of my responsibilities, falling clearly as you can tell in the domain of IT professionals, but it wasn’t terribly difficult and the whole exercise gained swiftness by orders of magnitude if I collaborated with the IT guys, so I figured why not. I can’t see how this isn’t exceeding expectations, unless the expectation is that I set up technology, do accounting, trade billions and serve chai and butter-toast to everyone while they play Oprah in the comments sections of Humans of New York.

So, yeah, it’s all looking good. Bonus and promotion announcements are still a month away, and my match dot com profile is already half-ready. In fact, what’s pending is just putting up my pictures, but, of course, that is the all important part. I do have a couple of nice pictures of mine from 2009 and 2010, and with just a little retouching, I should be all set. Uncle Baburam’s daughter Madhuri was very gracious about offering to “do amazing things with these pictures” at no cost, and although I’m quite tempted to take her up on the offer, I think I’ll hire a digital makeover expert from U2RHot for eight hundred bucks. What can I say, I’m not fooling around this time. And Madhuri should be focussing on her studies, Kindergarten is a crucial class.

You wouldn’t guess it from looking at me now, but my first six months at Kruger Ganley were a dream. I started at this job on the 23rd of June 2009, about two weeks after the graduation ceremony, and exactly four months from the day I’d started dating Swati. It was a phenomenal year, 2009. Everything about that year was perfect. If I left a problem in an exam because I’d have no clue how to solve it, I would later discover that the problem itself had being scrapped for some trivial linguistic ambiguity in the way it was written. This ball I hit out towards the hostel windows on the second floor, while playing cricket on the narrow alley next to the building this one time, went straight to the singular window without a pane, thus saving me the huge fine I was going red with dread about as the ball made its way up the projectile. Nothing could go wrong. Nothing could go wrong in the crazy unbelievable way that I now regret why I never dabbled in gambling or sports betting while it lasted. If I were ninety years at the time and left for a morning walk in a fit of rebelliousness and trembled on a rock, cursing everyone I ever knew in my head in the microsecond I imagined I had left with me on this planet before I hit the ground, a hot, top-naked girl would have come running from the woods, stopped me from falling, and kissed me passionately for no discernible reason. God was that kind of kind. 

The day I got the job, I got myself five Park Avenue shirts, one for each weekday. I would show up at work early, and smiled at everyone as they came in, just as uncle Baburam had advised. People seemed to like me, I stayed late and got everything done faster, taking workload off other people on my team who had been here a little longer, and have since all bought yachts and mansions and left the firm. My boss was supportive and treated us often, but I have to say under a different boss I might have learned a wee bit more. Most of his mentoring revolved around giving me such illuminating pearls of wisdom as “It is what it is”, “You’ve got to do what you’ve got to do”, and “That’s what it should do, you’re right, but the reality is it does whatever it is it does”. He could spend an awful lot of words explaining things which were explained just as well in zero. During meetings was always on display his unique ability to talk for an hour about nothing except what he’ll be talking about for the rest of the hour, until the hour was over, and we exchanged pleasantries and left. Was this the secret to multiple Brooks Brother suits while paying for your kids’ piano classes at the same time, I always used to wonder. I only stopped when he was fired a couple of years ago. The new boss, let's call him Aurangzeb, has proved to be very hard to impress. In the last two years, he has only taken me out for lunch once. That was two months ago, at Suburban Tadka. At the restaurant, when the waiter turned towards me after taking his order and I was putting on my greedy smile, about to blurt out the most expensive dish on the menu, he butted in and ordered something for me entirely on his own discretion. “Just what I’d wanted”, I remarked heartily. The waiter gave me a look I will not go ahead and describe, before turning back to him, clearly aware of his only customer that mattered, “So how spicy would you like it, high, medium, low? Medium, I suppose?”
“Yes, medium.” said Aurangzeb.
“Ok, sir”, said the waiter and began to leave, when he was stopped again.
“Wait, wait, wait. Actually, do very medium. Infact, very, very medium.”

That is not the sign of a man who doles out promotions easily.

Alpenliebe

For the fourteenth time in as many weeks, Vasudev Bakhshi was faced with the question of what to do during the two and a half day long weekend, when the office admin girl, Alisha Bhatia of the blunt nose and domed forehead, began giving out candies at every desk with a cheerful, if shrill, cry of "Happy Friday", stopping at every desk, and before he knew talk of what everybody was going to do on the weekend filled the colossal yellow-lit hall lined with fifty thousand desks, or so they seemed to Vasu, who, obese as he was, had been running the tip of his index finger along the periphery of the opening created between two ridiculously stretched buttons of his grey linen shirt, and wondering what it was that he used to do with his weekends during the summer four months ago when Sonia, his wife and a professor of Geography at the University of Bundelkhand, was home for the vacation and, to his surprise, he couldn't remember anything of import, neither any elated partying like when they were both collegiate and hungry for each other's touch all the fucking time, nor, thankfully, any crazy, viscious fights characteristic of the year before when she was still working in the city, although he did recall a certain Aarohi's phone number scribbled on the last page of one of her books, a number that he had committed to memory with a suspicious, foreboding fear, a fear that it wasn't Aarohi, but whoever Sonia took her increasingly numerous cigarette breaks out on the backyard to probably converse with, always in what he imagined a voice so low her own shadow couldn't overhear, but why o why o why couldn't she have talked about it, what is the worst that could have happened? Little Ridhi wouldn't be screaming in her bed every night, and I would at least have been able to eat this candy.

Sid Shoulda Said - Part 1 - The Short Term Goals of Sid Chat

We broke up three years ago, Swati and I, but I have to confess I have been thinking about her a lot lately. Being a man of indubitable character that I am, however, I won't call her. I strongly believe that it is very cheap to constantly call and text your ex-girlfriends, especially when, as in this case, they categorically cut your calls upon feeling the faintest shadow of your number upon their phone screens the fifteenth time they see it in a day.

My ex-girlfriend, the one who, some would say, I obsess excessively about, was the proverbial woman of substance, intelligent and driven, and I the typical man of substance abuse. She was also conventionally good-looking, had curls, big eyes, exquisitely shaped, soft lips, and a slim, affable personality. I knew other guys wanted her. I have no idea why she ever decided to date me with my flowing nose and fluffy arms, even though I admit my hair were the stuff of many a man's envy when shampooed. But that was rare, and besides, my teeth are a shade of color between yellow and green for which there is no name yet, as no other specimen of said color has ever been found in recorded history. She, on the other hand, brushed and flossed twice a day, and visited the dentist often. Her nose was pointed yet smooth, very much like a carefully sculpted nose made by a meritorious master's degree student of nose sculptures. In contrast, my own was made by an underpaid fast-food worker, who, when given this unexpected, unseemly task of making a nose, said what the fuck and made another samosa.

So I can only guess that she misconstrued my ugliness for my nerdiness, and gave in to her sapiosexual tendencies. I forget what the modern expression is, way out of my league, I suppose? But it wasn't the fact that she was physically attractive that drew me to her initially. It was that she was a girl, who would talk to me. That really was all. To really understand my obsession these hundreds of years later, you have to consider what had been happening with me in the years before.

I was precocious. As a toddler, I was already challenging the stereotype the whole world had been cooking up for ages about how all kids were cute. I do not remember a great deal of those days, but I do remember being constantly passed from one eager pair of arms to another reluctant one, before the latter would begin a frenetic search for the next victim. I am often told by my parents that I was the miracle kid who never peed his pants. Little do they know it was because I was so embarrassed already, I couldn't afford it. When I grew a little bit and reached the age when children start thinking they know shit, I realized that I was, after all, at least, a real funny dude. That was a big respite, I have to say. Every morning when I walked into the class, my classmates burst into instant laughter. I never quite understood, though, why they would hide my tiffin-box and leave chewing gum and pins on my chair and fail to tell me. To tell you a little secret, I never enjoyed that part as much as they thought.

But enough about me. I have to do something about this obsession with Swati if I am to have any hope of getting promoted. I have been slaving away for 4 years as a Junior Analyst at this bank, and each passing year that I find out I did not make Senior Analyst, the guilt of being a total failure gnaws at my chest hair. Before uncle Baburam, who was then a Senior VP also at Kruger Ganley, set me up for this job interview with an effusive recommendation, I had been looking for a year already. For a long time before that I had struggled with the question of what to do with my life. Everything I tried always felt either too easy or too difficult. Nothing was just the right level of challenging, except probably getting naked, but I've got to admit I could never have monetized that. Anyhow, as soon as I started working here, I knew it was for me. Plus I was over the roof that I had money to spend on my dates with Swati. Here I go again! No more talking of Swati, that bloodsucking bitch.

I have to get promoted this year. I promised myself I will only send out matrimonial ads when I can mention I’m a Senior Analyst. My brother thinks I’m a moron for wanting to mention Senior Analyst, which, he says, is an oxymoron of a title. Yes, he says things like that ever since he interned at this literary magazine, Intellectuelle, that sells exactly as many copies as it has employees, because there are only as many people in the world as pretentious. “What are you trying to tell girls, that you are the senior-most junior-most person in the bank?” he goes. What does he know about online matrimonial MO with his average build and his face that is not remotely annoying, and who the hell awarded him the pedestal to consider himself qualified to pontificate his literary hogwash over me. Besides, isn’t Junior Analyst a pleonasm, smartass?

By the way, I’m Chaturvedi, Siddharth Chaturvedi. Get used to the name, you will be hearing a lot of me. If there is one thing about me everybody strongly agrees with, it's that once I start coming, I keep coming.

The Spiral of D

Cry,

and think

how there's noone

to stop your crying,

but if there is, think

why're they not pained

when you cry like this,

but if they are, think

why are they not

pained enough

when you are,

and cry. 

Albert Camus, Franz Kafka, Jagjit Singh

Some of you who have stuck with this blog for a long time, assuming there are some of you, would know that not too long ago (well at least until 4 years ago, anyway) I considered literature my foremost calling, but more importantly, my biggest purpose. All that had started after a particularly mind-blowing experience of reading Camus' 'The Fall' on a particularly hot evening in 2006, a day before my Fluid Mechanics finals. Ever since, I have maintained that that reading, and the ones that followed, had a great role to play in me coming into being as an individual, the kind of person I grew into.

And, and Jagjit Singh with his renditions that emphasized poetry before musical pyrotechnics unlike his compatriots even in the relatively thoughtful rarified world of ghazals, was to me divinity itself.

Kafka and Camus are no longer personal heroes, but just a couple of NoSQL technologies I try to get fluent with.
JS now refers not to a personal hero, but the almost as admirable JavaScript.

Ah, life's changes.

 

A room of one's own

In this very week, 17 years ago, I moved to a new place. It was a huge change. I moved from New Delhi to, well, New Delhi, but in my own little life it was a development of colossal proportions. The new place was, well, a mile from the old one, but it was still quite honestly a huge change. I don't live at the said new place anymore. Haven't for a very long time in fact, but it was quite something, the giddy euphoria I used to feel back in those days at the prospect of having my own private place, a den, as it were, a room of my own. I was eleven years old, and wasn't doing any of the things that people most require a room of their own for. But, anyway, those were the days.

Also, these. These are also THE days. But let's come to that later.

Since I believe the blog should betray continuity of some sort, let me fill the large absences on my blog with a quick recap of my whereabouts since I seem to have shifted base frequently in the last couple of years. After completing my education in Pittsburgh at the end of last year, I spent more than two months at my brother's place in Cleveland, traveled to Florida for a bit and have since been working and living in Princeton. Now that the ambiguity regarding my whereabouts has been resolved, the point of the post is that my parents flew in here from New Delhi a few days ago! I just can't remember the last time I had felt this happy. When I graduated? I have to say that was the culmination of 17 months of the hardest I ever worked, but, no. When I found a job? That was a massive relief, because it immediately freed me from worrying about paying off a gargantuan debt, but, hell no. What are some other candidates? When I first fell in love, that was close, but no, not really quite there.

Until a few days ago, I used to doze off on most weekdays at around 9. It was a major departure from my grad school days when I was wide awake at school at 3 AM everyday but for some reason, it felt as if after an eleven hour workday starting at 7 AM, and watching a nominal amount of TV, it was the a natural and even necessary thing. It doesn't help that I know, in a non-professional way, a sum total of zero people in the city I live in. Besides, you've got to get up on time, right? Wrong, apparently. The thirst for sleep vanished as if by magic, and cheesy as it sounds, I feel so effusively happy reliving my teenage days of watching news on TV every night with dad that it is even somewhat embarrassing to admit. So effusively happy, secretly. Also, again, as if by magic, I who could not bear to stay awake until last week can't possibly go to sleep now, and feel like I have all the energy in the world to cook my parents dishes I've learned to cook over the last 2 years, drive them everywhere and be the the world's best bad tour guide.

I must have been foolish when I was 11. Or I must be foolish now, that the thing I least want, almost dread, is a place of my own, even though I can no longer say that I amn't doing any of the things that people most require a place of their own for. Yes, yes, I admit I had begun watching Bigg Boss. But they're off season anyway.

Lost on the sofa

I watched this travel series today, Stephen Fry in America, all episodes in one day. It was an amazing experience. The sheer geographical and cultural diversity of USA blows your mind, as does Stephen Fry's unmistakable charm. I have been watching a lot of travel shows lately, one other that I liked greatly was Happy People, a comparatively slow moving chronicle of the trials and tribulations, and most of all, unfading happiness, of the people that inhabit the Serbian lands in north Russia: the Serbians and the Russians. I could write long rambling posts on my experience of watching each of these two programs, but continuing being the lazy ass that I've been lately, I'd just briefly state how bloody significant I increasingly find it is to travel. And to end by the silvery consolation that watching travel shows, since not everyone can hope to travel so much while keeping up with their myriad, uncanny responsibilities, watching good travel shows can sometimes indeed be a great option.

Empathy as skill

Today, on linkedin, I saw an article headlined "Why empathy is your most important skill." I thought it was wrong on so many levels, I wonder where to begin. Also, I have no motivation to write long posts. So I'll just record that while I have nothing against skill-building, in fact it's a most noble activity, but the moment you start thinking of empathy as a skill, you miss the whole point of it, and basically guarantee that you probably won't be acquiring any, and stand to lose some of what you were born with. 

Momentous day

While recording only the third entirely sleepless work-night of my life last night, I submitted my last homework assignment at 4:30 pm today, exactly 24 minutes ago. Done with the last one. Well, hopefully.

Confession

For the last two months, I had been studying very hard. This mini-semester is famous for its insane inherent course-load, and to make matters interesting, I decided at the beginning of this mini that I would, on my end, also cover 3 full-semester CS courses in addition to my own course-load, during this mini-semester. I was working my way towards this fairly satisfactorily, and more often than not, the pain was sweet. Then yesterday, when I was coming to the college from the bus, I realized the bag was unchained when I got down, and the two registers of notes I had made over umpteen stolen hours on these 3 additional subjects had fallen from my bag. I lost them. Now it may not sound like a big deal to a distant listener, but to me, they were my single most valuable physical possession. I was really sad. And I could not concentrate on my studies the whole day yesterday, and the half-day that has passed today. It is not the first time something like this has happened: during the days I fancied myself a short-stories writer, my computer once crashed taking with it my twenty odd stories. True that I should have been careful. But, anyway.

The Unknown Man.

He's all about the things he doesn't talk about.
He is puzzled by his irrationality, because he (thinks he) can identify it.
He works hard. He doesn't know what for or why.
He is self-effacing but not not self-centered.
He (thinks he) can explain yourself to you.
He can explain himself to you.
He can not explain himself to himself.
He loves some people, who can be classified into those who don't know this, and, those who don't want this.
He likes walking on unknown streets.
He wants to be remembered by those who don't remember him.
He wants to be forgotten by most who do remember him.
He likes studying probability and liked studying philosophy.
He can kick his own ass.
He can not kiss it.