Some Mongo Fun

Posted: November 23, 2013 | Author: | Filed under: Learning, Software | Tags: , | Leave a comment

I’ve been doing “MongoDB for Node.js Developers” course from mongodb university and I’ve really enjoyed it. 5 weeks are over and hopefully the last week is not too bad. There are also courses for Java and Python developers that I plan to enroll for in future. I’m already very comfortable with the mongo shell and the other two courses shouldn’t be too bad, especially as I’ve been using Java on and off for a long time and Python is a fairly simple language.

Covered so far :

-Introduction

– CRUD

-Schema Design

-Performance

-Aggregation

 

Some of the more interesting things I see compared to relational databases:

– Documents are in JSON and in mongo shell the syntax is javascript.

– Need to think of document as well as code in javascrip objects.

-Aggregation is an interesting concept in mongodb – it’s like pipe in unix – you can sequence steps e.g. restrict based on certain criteria , then group on something else followed by sorting.

I didn’t use any IDE for this course – mostly vi and later on Sublime text editor. I see that Robomango is an IDE that can be used but I didn’t use it yet.

Also, instead of typing in the shell, the code can be written in a javascript file. The only requirement is that if you’re using a database other than default – ‘test’, make sure to switch it to the database.

use blog;
db.posts.aggregate(…..

 

I’ve put week5 examples at the below link :

https://github.com/viksingh/mongodBAndNodeMisc/blob/master/week5

Advertisement

Learning how to learn

Posted: November 19, 2013 | Author: | Filed under: Learning | Tags: , | Leave a comment

Learning how to learn is perhaps the most important capability in today’s world. However, the irony is that in some ways we have become our own worst enemies by being distracted and not engage in periods of deep learning. I’ve been on a 9 days meditation retreat once and the only thing I find kind of similar to is when I’m trying to solve a problem – I could be programming , debugging or sometimes thinking of abstract ideas.

The things that have personally helped me:

– Turning off noise : I used to be a huge news fan once with google news keeping me informed. However, with time I’ve realised that the world doesn’t stop churning even if I don’t follow news. This includes twitter, facebook ( I still have an account but have logged in very infrequently when I’m very tired ), television and reading best sellers.

– Focus on few things that matter: Health is a big things and that includes all aspects of keeping oneself healthy.

– Create a technology radar: Thoughtworks has a technology radar – it helps to create something similar for the learning process. What technologies I want to become better at ? Add to my portfolio ?

– Make it consistent: Watching energy levels so that I study when my brain is fresh and relaxed. Trying to learn when the mind is tired is not very helpful. Balanced diet , exercise and regular sleep is very important. Further, if I feel my brain getting tired , I shut down and just take a mini nap – it’s very effective to freshen my mind again. We humans have a natural rhythm and try to flow with it. Practice with a goal regularly will make anyone much sharper.

I’ve applied different strategies for different products :

– Find a good video tutorial on youtube / pluralsight : I’m optimistic about future of learning as we’re finally reaching a time in history when sending bits of video has become really affordable . This has created sites like pluralsight to become even more popular and of course youtube has videos over almost anything. I still prefer pluralsight as youtube can be a dangerous terrain – getting the information can be iterative.

– Companies have started with their MOOCs – SAP, 10gen ( MongoDB ) .

– Many conferences / sites publish their own videos: Includes oracle / sap / redhat / google / apple that I have used in past – typically in the developer Techeds , they cover what’s coming and what interesting has been built.

– Books: I don’t buy dead tree technical books anymore ( for the practical reason that I stay in a small apartment and it’s easy to carry kindle) but ebooks from SAP press , Oreilly , Manning Publications , Pactket Pub,InformIT , Apress etc – All have books in ebook formats – except perhaps for some of the old ones in case of SAP Press.

– Official product documentation: This is a joy to read, is most accurate and descriptive but requires more time and typically follows some level of familiarity with the technology.

– RSS Feeds : Once you’re already up to speed ( or comfortable at some level ) you can check blogs as RSS feeds.

When it comes to other areas, especially philosophy I stay away from new books – I find it more interesting to read books that are at least some decades if not centuries old.

Useful Unix commands

Posted: November 18, 2013 | Author: | Filed under: Software | Tags: , | Leave a comment

Useful as reference:

echo $BASH : a path gets displayed ‘/bin/bash’

ls -la /data ( display in the sub directory)

ls -R (shows more details – recursively)

/ used in man search

ctrl p / ctrl n ( command history)

cat <filename>

less <filename>

search with /

reset (if terminal set by any binary files)

file <filename> : will give info about file

open <file>

rm <file>

mv <filename> //rename

mkdir a

mkdir a, b , c, d

touch <filename> //create a new file

open . (opens current dir in finder)
cp file.txt . (copies to current dir)
cp subsir/file.txt demo.txt (copies from subdir and uses the name demo.txt)

cp -R ( copies dir as well…)

mv <file> targetpath

rm <filename>

rmdir <dirname>

Safety switch -i

It will ask before doing dangerous things

cp -Ri

mv -ri

ls a* (All files strating with a)

ls -d (Lists only directories)

? match single char

[abcd] match anything that has a b c or d

[^a-c] not matching a through c

rm file[234] ( will remove file2 through file4)

rm file[^234] ( will remove anything not in 2 3 or 4)

rm file?.doc ( will remove all which has a single character)
touch {a..c}{1..3}.txt (creates 9 files)

ls -lR > listing.txt (output to a file)

ls -lR >> output.txt ( appended to the file)

cat > story.txt ( usefult to create files)

echo >> story.txt ( same like story ..just appends)

Pipes

ls | lesss ( combine ls and less)

grep text <filename> ( will find text in filename and it’s occurrences)

cut -f 3 oscars.tsv | grep 4 | wc -l ( Piping counting number of movies with 4 oscars)

Substitution

echo “hello $,(whoami)”

Some thoughts on software development

Posted: November 18, 2013 | Author: | Filed under: Software | Tags: , | Leave a comment

For whom to write software :

– Users : Please think of the fellow human who’ll be using the software. Let’s make his life better.

– Fellow developers: Compliers can parse the worse of our code but software is written in a certain way for the fellow developers – someone showing us our code after 5 years shouldn’t cause us embarrassment. Or let’s try to minimize it .

– Company : Someone is paying you to write this , to solve a certain problem. Think of them as well – how can you add more value. Many a times, it can be different than just writing code – a suggestion about what’s possible.

Things to consider in mind:

– Don’t cause  bugs.

– Think of the larger architecture. Try to understand how various things interact with each other. I have found it to be key in problem solving – having a higher level complete picture makes us focus on areas where handshaking happens or corner cases arise where most of the difficult issues arise.

Anti fragile software

Posted: November 18, 2013 | Author: | Filed under: Misc | Tags: , | Leave a comment

Software especially critical software ( as used in integration layer ) can benefit from Taleb’s triad of fragility, robustness and anti-fragility.

In short:

Fragile : If a thing loses from randomness e.g. a glass falling on the floor and breaking down.

Robust: If it works normally . Most things do when no stressor is applied.

Anti-fragile: It gains from stressors. e.g. Human body when we exercise.

Software: Taleb didn’t touch software but given how important it is becoming in today’s world – some degree of redundancy will help to cater for situations when things go wrong. For example, if you just have a middleware bus and it goes down – what happens next ?

Netflix uses this concept literally – it has a chaos monkey. According to them – “We have found that the best defence against major unexpected failures is to fail often. By frequently causing failures, we force our services to be built in a way that is more resilient.”

In college, I had class on TMR ( Tripular Modular Redundancy ) .  TMR is a fault-tolerant form of N-modular redundancy, in which three systems perform a process and that result is processed by a majority-voting system to produce a single output. If any one of the three systems fails, the other two systems can correct and mask the fault.

In TMR, three identical logic circuits (logic gates) are used to compute the same set of specified Boolean function. If there are no circuit failures, the outputs of the three circuits are identical. But due to circuit failures, the outputs of the three circuits may be different. A majority gate is used to decide which of the circuits’ outputs is the correct output. The majority gate output is 1 if two or more of the inputs of the majority gate are 1; output is 0 if two or more of the majority gate’s inputs are 0. The majority gate is a simple AND–OR circuit: if the inputs to the majority gate are denoted by x, y and z, then the output of the majority gate is:

xy \lor yz \lor xz

How it is relevant to modern software development ? Software is becoming more pervasive and complicated with time. We can learn from nature how it has created redundancies – the system may not be the most efficient at a given point of time – there is redundancy.However, over time it is definitely more stable.

Tale of two MOOCs( Massive Open Online Courses):

Posted: November 16, 2013 | Author: | Filed under: Learning | Tags: , , , , , | Leave a comment

MOOCs have started for courses from Universities some time back and I have used the one from Stanford for iOS development. Thanks Prof. Hegarty – If I had to do it again, I won’t necessarily do it in the same way ( it turned out to be very inefficient for a beginner who has a day job doing something else). The course definitely is good but has a relatively steep learning curve. It starts to get difficult if you have a day job.I found books more helpful to learn the fundamental concepts of Objective C.

However, it seems that the time has finally arrived for companies to start offering MOOCs for their development platform. Being in the SAP world, I have enrolled and finished the following:

From SAP:

– Introduction to Software Development on HANA : This was the first one from open.sap.com and it helped that it’s for HANA – the cool kid in SAP world these days.The course was well structured and the fact that it had practical tips on how to fire up AWS instances, create AMIs to save costs etc was very helpful. It was more HANA general rather than focusssed on ABAP on HANA.In the beginning it was a bit challenging ( especially week 2) but then it picked up a very good pace and I enjoyed working through it. Apart from HANA, the biggest message was – Ignore JS at your own peril.

-Mobile Development on SAP Platform : I have already worked on a project for an iOS application using SMP, supported it around for a year and have learnt a great deal about the differences between a mobile application development project compared with traditional application development projects.This course still proved to be very useful for couple of reasons:It had SAP Netweaver Gateway providing services in oData protocol for the backend.The course was more for developers who’re already able to build mobile applications and it’s focus was on presenting SMP as a MEAP that can be used for enterprises. I’m satisfied with this course.

– HANA Cloud Development : I have been doing a lot of Java development, especially in the last 3 years and I was happy to see JEE skills being useful here. The course creates a typical JEE application but using SAP HANA Cloud as a platform. I’m mid way through the course and have learnt some new things – e.g. document management service in a cloud application. I see cloud as the platform for future. Not only in SAP but also outside it – redhat is pushing for it’s ESB on cloud.

From MongoDB University:

I have signed up for MongoDB for Node.js developers course from MongoDb University. There are two things that I’m learning in the course.

– ALl my life has been with relational databases. This is the first time I’m doing any kind of work with a non relational database. Initially I found the idea very weird – what happened to schema ? However, thinking about iterative development – it doesn’t turn out to be a bad idea at all. In some ways, the schema is in the application and a collection ( entiity in the SQL world) can have different kind of data co-existing – before and after a patch to add a new field as an example.

– Node.js : I have done the pluralsight course but it’s good to apply it for a practical blog application.

Book Review: So Good They Can’t Ignore You

Posted: November 2, 2013 | Author: | Filed under: Learning | Tags: | Leave a comment

I really didn’t plan to read the book until six hours back – happened by serendipity and I’ve enjoyed reading the paper version of the book. Key Ideas:

– Passion Hypothesis : Steve Jobs asked us to find our passion but the truth is we don’t start out in this way. Neither did Steve knew what he wanted to do early in his life. We will be happy if we can just figure out a job which will make us happy. Carl describes it both wrong and dangerous – it’ll lead to bigger disillusionment.

Be so good they can’t ignore you : A quote from comedian Steve Martin about what is needed to make a life you love.

The craftsman mindset: An approach to working life in which you focus on what value you can provide to the world.

The passion mindset: The author describes this as your focus in which you focus on what value your job is offering to you. Perhaps this is responsible for a lot of dissatisfaction people have from their jobs ?

Career Capital : Have the skills that you have that are rare and valuable to the working world. This is essential for creating work you love.

The career capital theory of work: Don’t follow your passions but get good at something rare and valuable . This will help to generate the “career cash” – more than money that defines great jobs.This requires approaching work with a craftsman mindset instead of a passion mindset. It can be broadly defined as follows: –

– The traits that define great work are rare and valuable.

– Basic supply and demand law states that the rare and valuable skills you can offer.

-Having craftsman mindset.

– 10,000 hour rule : This was introduced by Gladwell and is deliberate practice for a long period of time.

– Deliberate Practice: This perhaps is the biggest learning I get from the book. I had an idea of what deliberate practice is but wasn’t really clear. It essentially means going beyond what is comfortable to improve. It might require multiple iterations to get something done but is the only way to train neurons – I find it very true in my own experience. When I started learning Objective C ( a lower level programming language compared to ABAP ), my brain wasn’t really enjoying the experience. I persisted by brute force. This is one of the methods described in the book. However, two approaches are possible:

  – Mark some time when you just decide to sit with the subject no matter how hard it is.

  – Create some goals and small quizzes to track sub area progress.

This also gave an interesting ideas of creating a weekly chart of hours spent in deliberate practice. In fact I find it very true intuitively. After a lot of hard work, brain definitely needs a break.However, it’s important to relax and not indulge in online distractions.

– Career Capital Markets:

Winner takes all : Typically seen in sports

Auction: It’s about the skill combinations .

Interestingly when looking at software development, enterprise software definitely looks like an auction – which is a thing I enjoy ( learning different but somewhat related technologies ).

 

Control : Having a say in what you do and what you do it. One of the most important traits to acquire.

1st control trap: A control that’s acquired without career capital is not sustainable. e.g. Asking to work part time in the first month of your job .

2nd control trap: You have acquired enough career capital to get some control over your work.

courage culture: The idea in many online communities that the only thing between people and happy careers is the courage to leave the expected path. It’s dangerous as it underplays the role of career capital.It has lead many people to leave jobs where they’re worse off than before.

Law of financial viability: When deciding to a new path, ask a simple question – are people going to pay for it ?

Mission : The author has some good insight here. It takes time for us to be clear about our mission. Till then, focus on the job at hand.

Adjacent Possible: Originally coined by Stuart Kauffman it explains the nature of innovation. The innovations are not really magical. At some point of time, it becomes inevitable that the innovation will happen.However, people need to get the cutting edge before adjacent possible becomes visible.

Little Bets: Rather than a big bang go for small little bets – try out iterations and seek feedback.

Law of remarkability: People who encounter it must remark and it should be launched from a venue that supports such remarking.

Key actionable learnings:

 – Have a craftsman mindset .

– Focus on creating value to the world.

– Deliberate Practice: Do it regulalrly – I’m already feeling my neurons rebel while trying to finish second week of node.js with mongo DB course.However, it needs to be continued.

– These ideas can be combined from Neal Ford’s techinology radar – Create list of technologies for different categories . The ones that can create value should be developed with regular deliberate practice.

– Personal email can be checked once a week and it’s a good idea to stay away from twitter.