Customizing WordPress to work like Confluence

The image shows a smooth transition from the blue and white confluence logo, which consists of two overlapping circles and a letter C, to the blue and white wordpress logo, which consists of a stylized W inside a circle. The image conveys the idea of changing from one platform to another, or integrating them together.

At Homeville, when we started active product development, JIRA and Confluence were our go to tools. As the technology team grew from a 4 member team to a 65+ member team, one of the things that became a slow burn for me was to see the monthly user-wise licensing model for both of these tools.

JIRA was free for the first 10 users, then for the next 40 odd users that we added, we were paying roughly 400 USD per month! Similarily, Confluence was free for the first 10 users, but at a team of 40 or so, our bills from Atlassian was matching our bills from AWS!

Shifting from Atlassian

One of the quickest shifts we did was to find an open source self hosted version of JIRA. OpenProject was the perfect solution. The benefit of self hosting these systems was that access to all the proprietary analysis and research in the form of features and requirements was now within our AWS account and thus in a much more secure environment. The added benefit of localization within the country also meant any form of regulatory compliance was also much better aligned.

It took some scripting, but using a python script to connect to JIRA and OpenProject (OP) both, we were able to import the projects, tickets and sprint informations from JIRA to OP.

Wiki or independent system

One of the key decision that we had to undertake was to see if the documentation was to be done within the OP, or was it to be hosted separately? One option was to continue with Confluence albeit under a smaller user base.

We finally decided to try things with a self-hosted WordPress. However, we did tweak quite a lot of things, listing all of these.

Customizing WordPress

Going into this project, I knew that this would be a continuous evaluation and test work. Sort of a rinse and repeat exercise. Hence, choosing WordPress was the obvious choice.

Custom Taxonomies

We setup PODs because of its ability to quickly add custom post types, custom taxonomies and easy relationships between the two. Using this we could easily keep a separate taxonomy for our credit platforms and systems.

Authorization and Access

Since we were on Microsoft o365, we setup an active directory plugin to enable tech team users to login using their o365 credentials. This also seamlessly created their user ids on WordPress and enabled access without the trouble of invites and importing users.

We also wanted a login gate to enable authorized access to our content. Thus a simple membership plugin to restrict content access was setup.

Similarly, we also setup a plugin that disabled the REST API so that the content wasnt available over unauthorized REST API calls.

We also setup a NACL rule on our AWS to prevent access to the system outside the office premises.

Integration with OpenProject

We also setup easy shortcodes that would pull data from OpenProject agile sprints and display those within a post, allowing for product teams to create east release notes followed by the sprint backlog with links to the OpenProject stories.

This was done using a custom plugin integration that we wrote. Obviously this could not have been possible without OP not having a REST API.

Document Templates FTW

This entire process would not have been successful without the new WordPress editor and its ability to create document template types within the post.

We took our most commonly used Confluence templates and set them up exactly in the same manner in WordPress. This enabled our product teams to seamless shift from Confluence to WordPress.

The fact that most discussions can also be captured as minutes with the exact time of publishing as the time when the meeting occurred also meant that we could also have a temporal view of our product development.

Thus with these simple and easy tweaks we shifted from Confluence to a self hosted WordPress.

Reduction in Stamp Duty rates

Stamp duty in Maharashtra

In a move to bolster real estate sales, the Maharashtra government has announced a reduction in stamp duty rates of up to 2-3%.

As a new home buyer, this is an opinionated piece and somewhat of a warped perspective. However, I will try to be as objective as possible and hope to give enough citations to qualify my stance.

What is stamp duty?

Stamp duty is the additional charge that you will have to pay if you are buying a home anywhere in India. Depending on the state you are in, this stamp duty is payable at different stages in the home buying journey.

In Maharashtra, the stamp duty is to be paid upfront when you are doing the home down payment. In other states, such as Karnataka or Telangana (I am mentioning these because these are the two fastest growing states in terms of real estate) this stamp duty is to be paid on possession.

Why is this so important?

Well, most people end up saving for buying their first homes. Unless if you have access to super awesome payment plans and offers such as the home down payment assistance of HomeCapital, the majority of their savings end up being spent for buying that first home.

A stamp duty is usually levied on top of the agreement value. So in Maharashtra whenever you buy a home, not only will you be paying the usual 5% GST, you will also have to pay a 5% stamp duty as well. This pretty much puts the cost of the home at 110% of the agreement value. This is not even factoring in the cost of the broker, the registration fees, the home loan processing fees. If you add that up, the cost of the home is often 115% of the agreement value.

Stamp duty is 30% of this chunk. This chunk of expense is usually not visible to the average home buyer, until the point of purchase. That means you realize that you have incur additional expenses when you commit to buying a home.

Reducing this stamp duty from it’s 5% to 2-3%, the Maharashtra government has reduced the overheads of home buying.

So … what is the actual impact?

This is the question that a lot of us are asking. The actual impact if you are purchasing a home anywhere in Maharashtra, is a 60% reduction in stamp duty (that’s 2-3% of the agreement value). So, if you were to purchase a home worth 1Cr INR (roughly USD 140,000), then the net benefit you are getting is 2-3 Lakhs INR (roughly USD 3000-4000).

Would this impact real estate sales in the long term? No. In the larger scheme of things, this is but a drop in the ocean that’s not the painful part. In the smaller scheme of things, there might be some speculator transactions hoping to cash in on the “opportunity”.

Having just finished the worst quarter in the last 20 years, the sales are bound to rise. As the industry slowly recovers to its pre-COVID numbers, this small respite is a precisely that. It’s a small reprieve and pretty much nothing else. After 10 years, no one will remember this move, however, if this move were to solidify into the norm … then this would be interesting to see.

Conclusion

I think the government needs to look at the larger issues of access to affordable capital. Granting a small reprieve is not really an incentive to the industry.

The short term transaction upheavals would be an issue, and this hurts the same industry more than actually helping. However, most state governments have historically shown to be myopic and short sighted .. due to the nature of their terms and I cannot really fault them for this. So, like all things in the past 4 years, this shall also pass.

6 months of lockdown

As I write this after nearing the 6 months mark of lockdown, I cannot help but think at looking back at how things have changed in the last 6 months or so.

  • Work from home is an accepted norm with remote working at an all time rise. The organizations that could slide into this mode of working have also started realizing the benefits of allowing teams to operate from home. Any teething troubles that were there have been ironed out and I am see teams of all functions coming together on Zoom/Hangouts and making it work.
  • Reverse migration has started. A lot of this working class who can work remotely has opted to move back to their native places. Just to give an example, out of my team of 8 – only one has chosen to stay in the city … the rest are safely back at their native places across the country.
  • Internet penetration and mobile services are at an all time high. The demand for Jio has never been higher with this working class scrabbling to ensure that they have steady connections at home. I see this audience’s demand in Tier-2 and Tier-3 cities ensure that brands and the government focus on building out the infrastructure in remote cities.
  • This would lead to some normalization between demand and supply of all goods across higher and lower tier cities. Take Mumbai for example … in the suburbs or in Mumbai proper, it is hardly a case when you see an electricity outage. As you go outwards, you will start seeing specific load shedding hours and schedules. In the Raigad district, there is atleast one day a week when there is no electricity. As the working class goes back to these cities, either the demand for inverters will go up or the respective local governments would be petitioned to increase the quality of lifestyle.
  • Environment conditions across all cities have drastically improved, the Mumbai air feels cleaner, cooler and taking a walk doesn’t seem oppressive.
  • Organizations whose engagement models involved a lot of physical interaction have started discovering alternative methods and workarounds. Dentists have started using full-body kits, delivery boys have established clear package hand-off protocols, restaurants have started opening up with lower floor space utilization.
  • Cost of basic services and commodities have slowly increased. An annualized inflation of 15-16% looks to be on the cards and the common man is going to bear the brunt of this. Any initiative the government is going to take is only further going to exacerbate this.
  • Industries that have been doing well since lockdown –
    • Food Deliveries
    • E-commerce
    • Agri-tech
    • App enabled services
    • Edtech
    • Fintech
  • Communication apps are at an all time high. Zoom has made it to the top 10 websites in India according to Alexa.com
  • OTT platforms are raking it in with a lot of the younger audiences looking at their smartphones for entertainment. Since there haven’t been any theatre releases, all the movies that were scheduled to be released have started being covered on the OTT platforms. A quick glance at the above list by Alexa informed me that Netflix, PrimeVideo and HotStar were all in the top 20.
  • Big tech firms are going all out to change the way things are. Google pretty much gave all schools free access to Google Classroom. Both my children are using this for their new term this year.

As things start settling down from this massive change in life, I see a resilience being shown by businesses as they start figuring out a way to live and thrive in this economically challenging environment. As a technologist, I see a large need to automate a lot of business processes to keep the wheels of the industry turning.

This is what will keep the world going round.

Building your Custom Connector on Google Data Studio

Building your Custom Connector on Google Data Studio

Disclaimer – This is going to be a slightly technical post. If code scares you, then I’d suggest you skip this one. However, if data excites you, read on fellow adventurer!

When Google launched Google Data Studio, I had written an in-depth post about how one can create dashboards in Data Studio and some of the easy to use dashboard templates that Data Studio has to offer. As the product evolved, one of the most powerful features that this product had to offer was the ability to create custom data connectors to your own datasets.

What does a custom connector do?

A custom connector enables a user to access their own data source within Google Data Studio. Let’s take an example of a marketing research associate who wants to present her findings. One approach she could use would be to put all that data in Google Sheets, and then use one of the in-built connectors.

However, what would she do if her data set was large and does not fit in Google Sheet or Excel? Or if her data set included multiple surveys which are inter-related to each other?

What if this data was in a database, or available as an API? This is where the custom connector for Google Data Studio works.

I wrote my first connector a year back, and I had to do some digging around. I thought that I should pen down my notes so that more people can do this much more easily. Here are my notes for the same.

Building a custom connector

Before you jump into the implementation bit, know that this is based in JavaScript and you need to be comfortable with Google App Scripts. It’s okay if you do not know this, but JavaScript is a must.

Google has official documentation on the developer site of how to build a Community Connector, this is a pretty good resource to start. It has a step by step video and instruction guide as well.

Let’s look into what makes different parts of a connector, here is a link to a sample connector code on Github.

Community Connector sections

Each community connector is a separate Google Apps script that you deploy using Google App scripts. The connector is in itself made of the following sections –

  • Configuration section – This is to flesh out all the meta information about the community connector. Use this section to take any inputs from the user e.g API Secret and key if you don’t wish to store this in your code.
  • Authentication section – This is to authorize the app script. If your data is lying behind a secure mechanism, then use this section to authorize the script to access the data. This supports OAuth2 as well.
  • Schema section – This is used to define the structure to the data you are importing into Data Studio. Use this section to outline which fields and what data types are they. You can also add more information on what kind of aggregation do you want this field to be have (Sum, Average, Min, Max, etc).
  • Data section – This section is used to fetch the data that you are importing. Use this section for data validations or if you want to do any last minute data tweaks (e.g date conversions from string to date).

That’s all there is to it. Now, let us get into the actual flow of the script.

Connector code flow

When you are writing your connector, be sure to go through the developer reference first. In your script, you will have to include the following functions –

  1. getConfig() – this returns the configurable user options for the connector, this will be shown to the user when the user is adding the connector to their Google Data Studio accounts.
  2. getAuthType() – this is the function which is used to check if any authentication is required. If OAuth is set, then the community connector interface would check for the OAuth details
  3. getSchema() – this returns the schema of the data that is being access, this will be shown to the user when the data is being explored (where we can see the dimensions and metrics).
  4. getData() – this is the function which is used to access the data, the data format that is expected is outlined here. Normally, it is advised that the programmer write a separate function for fetching the data, a post processing function for setting up the return values, and finally call those in the correct order in this function.

Do note, that these functions will be called in the same order as they are listed. As long as you have these functions in your code, you have a functioning connector. Once you have this, you will have to deploy the code.

That’s it. Now, add this community connector to your Google Data Studio account, and make the reports you want to!

Life and opportunities post COVID-19

Life post COVID-19

It has been 2 months since we have transitioned to work from home and figuring our way out in this time. I noticed that lifestyle and working style of many of us has changed (perhaps for the better). Here’s a list of some of the highs and lows of life post COVID-19.

The highs

  • Work from home has many benefits. The time saved in transit for one. I have been saving 2 hours of idle time wasted.
  • No household help means more work, but it also means you can save on the household expenses because you were otherwise too busy.
  • You become more aware of the household chores, so does everyone else in the household. That’s a liftime savings of bad habits avoided. Not to mention that a lot of households will not go back to hiring so many household maids. We had 3 for instance.
  • Businesses have been forced to either work remotely, or to re-think their operational models. That has meant a lot of top management mental models have changed. Earlier where physical meetings were the first option (and sometimes the only option), now telephonic and video calls are working.
  • Since physical interaction between people is the way the COVID-19 is spreading, most businesses have figured out a way to work without this physical engagement. That has meant higher operational efficiencies, and also a focus on being more transparent.
  • A more conservative mindset is emerging in the society, with all expenses being more need driven than greed driven. This as meant less wastage and less consumption.
  • A large push has come for online education and self help videos. The OTT segment has really bloomed during this phase, with binge
  • The ecology is improving due to a massive cut in the consumption of fossil fuels. The hole in the ozone layer is now no more, that talks about the scale of the lockdown. What we could not as a generation manage in the past decade, a virus outbreak managed in 2 months.
  • Industries where digital enablement was missing are now the worst affected, and that’s where new opportunities are emerging. These businesses are figuring out a way to operate in post COVID-19 world. Some examples of this – real estate, financial services, banking, automotive, etc.

The lows

  • The economy is in an all-time slump with all non-essential services at a stand still. Getting them back to what they were before this crisis will easily take a year or two.
  • There is now one more excuse to discriminate and divide people
  • Physical exercise and activities for people who need them (me, my kids and parents) have diminished.
  • Non-critical healthcare and luxury item supply chain has been wrecked and acquiring specific brands has become difficult.

In the end

As a teen, I have played the hit series Fallout. I am surprised to note how fast we have shifted into living that way. The only difference is that this crisis is not nuclear, it’s a virus.

We have adapted to keep on living. What remains is how long this will continue and what practices do we keep as the world slowly comes back to normal.

Work from home the new norm

As if taken from a zombiecalypse movie, the coronavirus outbreak (COVID-19) has impacted the entire world in a big way. Without meeting people how does work get done?

How does the economy keep running inspite of the lockdown?

These are the questions plaguing a lot of business owners. Some have continued to brave the outbreak by declaring themselves as essential services. However, take the example of some obviously non-essential services – Real Estate, Information Technology, Financial Services apart from banking. How are these industries to function?

Finding a new mode of working

The obvious answer to this is to find a new operating model. Many organizations were going the digital, or were in the process of doing digital. Now, these organizations are most suited to survive the outbreak.

For pure brick and mortar businesses, transitioning to a new way of working is paramount.

This is where work from home comes into picture. This is not anything new and in the ITES sector, the work from home paradigm has been around for more than a decade now.

However, it was always considered as a secondary mode of working and never the primary.

A new operating model

With most businesses struggling to find a way of working online, more and more tools are being sought for online collaboration.

My wife, Dr. Harshaja who runs 13 Llama Interactive marketing agency created a quick video on some tools that people can use during work from home.

5 Tools for Work from Home by Harshaja Ajinkya

A new day, a new way

The human race is very resilient and always finds a way out. I hope that like all things, this too shall pass.

Until then, lets all herald the new way of working!

Working with markdown and gitbook

working with markdown and gitbook

For those of you who don’t know yet, I have shifted tracks to heading a tech team in a start-up. This firm focuses on helping first time home buyers with the largest hurdle in home buying, the down payment. HomeCapital is India’s first home down payment assistance program.

At HomeCapital, one of the immediate challenges that I had to face was to understand a myriad of requirements from speaking to the operations team, to the business analysts, to the developers, to some of the customers and even to some of our investors.

Since, the approach is that of a technology platform, it also means that the team had to start worrying about multiple systems all at once. Deciding to move away from one huge monolithic system to a micro-services based architecture was natural.

How does one manage loads of Micro-services?

A major challenge with a spread of micro-services was that the management overhead of systems went up. Different services were in different repos, in different languages and hosted in different methods. Yes, there was an API gateway on top to present a uniform access method for all, but the code management and documentation was a challenge.

Thankfully most popular versioning systems have solved the code management issue. One of the first steps I initiated with this was using the README.md to quickly jot down what the service is supposed to do, and how it functions. This was created more from the point of a new team member who wants to get started with the respective service. You need to be comfortable with Markdown for this. I’ll get to markdown in a minute, but this was a great starting point for me to understand what a developer really needs in the documentation.

As a person overseeing multiple services, it was essential for my team members to quickly pick up the bare essentials and use the documentation available. Having a small entry point in the repo is a perfect way to give access without creating too formal a structure. My choice of working with markdown was made.

What is Markdown?

In case if you do not know what this is, then you mostly haven’t edited a wiki. Markdown language is a super lightweight language that allows one to quickly convert the text into a rich formatted document (such as HTML, PDF, etc). To read more about this, head on to the Wiki on Markdown.

Try practicing using Markdown for some time and you will realize its almost as simple as using notepad or gedit to take down your notes. It also helps you to create a more complex structure and is super flexible for future use-cases.

Generating a usable README.md

For those of you who want to try this out, hop on to Make a README and see the basic placeholder sections needed to make a developer friendly file.

I had by this time quickly written these files and was happy that at least I had some formal documentation available in a system that was fast growing. A side note here – <rant> In most rapidly evolving systems, people often take decisions that they regret later on. This technical debt although is meant to be avoided, but often it just can’t be avoided. As long as you are willing to come back and clear the debt, it’s fine. You could re-think your approach and do it faster in a correct fashion – but then you need to be a lot more mature and I just don’t see that developer maturity yet. This side note will need to be expanded into a separate post of it’s own </rant>

What to do with a cart load of README.md files?

Quickly, I had many individual standalone files sparsely connected to each other. While this was sufficient for a developer to get started, this did not fully cover the breadth and width of the system.

This is where my past experience of working with the WordPress India community helped. The community is building an independent document made of such .md files using gitbook. Gitbook used to be a CLI based command that you could install on your machine and use to build a developer website. This using the very .md files that I now had.

At the time of writing this post, the gitbook CLI is available on npm, however, do note that the site now talks about a version 2, which is not a CLI based offer but is more of a SaaS product with a freemium offering. You could also look at some other alternatives to do this, but the ease of use of the gitbook CLI is to be applauded.

How to get started with gitbook?

  1. Head on to the npm page for gitbook-cli and install this first.
  2. Create a new folder and in the console hit gitbook init
  3. Answer the questions and create your first markdown file
  4. In the console hit gitbook serve and in your browser go to http://localhost:4000
  5. That’s it

Core concepts

Keep in mind the following things –

  • The SUMMARY.md maps to the sidebar on the left hand side. This can be styled and the content of this file pretty much decides the navigation of your gitbook
  • gitbook is extendable through the config file – book.json, not just in look and feel, but also using plugins. My must plugins are – ["collapsible-chapters","insert-logo","image-captions","tbfed-pagefooter","copy-code-button","ga","sitemap","mermaid-gb3"]
  • Create sub-folders for different modules/services
  • Have a list of all entry points in SUMMARY.md
  • Maintain a CHANGELOG.md to have a history of major changes made
  • When a particular module becomes more complex, divide that into more parts and put those parts into nested folders. Do not forget to update the links in the respective .md files
  • Make the respective indents in the SUMMARY.md file as well

Building your gitbook

You can even host this somewhere (such as an S3 bucket or a static hosting). Simply execute the following command –

gitbook build

This will create a new _book folder in your gitbook folder. Host this as the static site.

That’s all there is to it. A simple and easy way to manage an evolving set of markdown files using gitbook.