Blog: Rajgopal Kishore Subscribe to this blog's RSS feed!

Rajgopal Kishore

Welcome to my blog. I wish to share best practices, insights and trends on business intelligence (BI). To me BI is about measuring your business, discovering performance levers and enhancing business performance. Effective BI is a closed-loop feedback system that learns constantly and is reoriented based on performance improvements.

Tools and technology are part of the solution but are not the solution in themselves. Too many organizations have all the right tools, technologies and technical skill sets but still fall short of effecting performance improvement.

This blog is about the problem-solving approach required to make BI impact business performance. My blogs share my personal insight gleaned by consulting with Fortune 1000 organizations and creating world-class SI practices. Some of the themes I write about include:

  • Gaps in current tools and technologies
  • Suggestions around organizational structures and skills
  • Making IT successful in BI
  • Client experiences - both good and bad

Join me in this endeavor.

About the author >

Rajgopal Kishore is an accomplished industry leader with more than 20 years of experience. He consults with Fortune 1000 clients around IT and BI strategy. He has jumpstarted and scaled IT/BI consulting practices at top-five outsourcing/system integration companies. His personal passion is to help clients realize business value from technology and outsourcing decisions. Over the last decade, Kishore has consulted on enterprise architecture, IT optimization, architecting complex transaction systems, performance assessments, IT strategy and BI strategy. While building consulting and solution delivery organizations, Kishore has relentlessly focused on listening to clients and providing solutions to real client needs as opposed to articulated requirements. In his last stint at a major IT outsourcer, Kishore felt a need to reorient team members to consultative engagements and, as a result, he created a game-based and case study-based consulting workshop. You can contact him at rkishore9@gmail.com.

 

The other day I had the opportunity to review a platform & service that purports to reduce the inventory of cash in the ATM network. Cash transactions and end-of-day levels were recorded and sent to an analytics engine. The information so collected is subjected to analytics algorithms and some manual analysis.

What happened when the analytics engine detected any sub-optimality? It flashed specific recommendations on the frequency of refills in each ATM, the bill denominations and amount at each refill. The analytics service is measured by the reduction in the inventory of cash in the ATM network, not by the number of reports that it generates.

 

Shouldn't this be the way Business Intelligence and analytics works anyway?

 

The tragedy of  BI / analytics today is that it is seen as technology to present information, and is often disconnected from operational systems. It rarely provides insights or recommendations for specific action.

Peculiarly, the biggest impediment to progress in not intention but organization design. I discover most services companies have flawed organizational designs - and hence progress in this area continues to be limited.


Keeping the above in mind, I have the following suggestions and ideas:

  • If you are considering BI and analytics in your business, centralize your data management and datawarehousing functions but embed analytics and BI into your lines of business function. For example, the risk guys should have their own analytics, the fraud their own, procurement their own and so on.

 

  • If you are an IT/Business consulting services player, differentiate the plumbing and foundations that are more technology centric from business intelligence and analytics that are closer to business. Consider creating Data Management/ Datawarehousing as a line of business focusing on foundations - data management, ETL, appliances, high performance computing, latency, physical data modeling and optimization. However, create Business intelligence and analytics as closely aligned to the industry oriented business units.

  • Consider business intelligence and analytics as an incubator for new ideas in your industry business units. After transactions systems have been perfected, the next set of performance improvements will only come from data. Remember - none of these great ideas will take off until you business units own them. So let them drive it.

  • Enhance your business consulting with the above capabilities. There is a huge capability gap in the industry. The gap is NOT on technology. The gap is around blending 4 elements to achieve enhanced business outcomes. These 4 elements are - understanding of the business, understanding mathematical modeling, visualization, and IT foundations. The best guys to "get it" are business consultants - I opine these guys are sharp and can learn what is needed to transform business as opposed to leaving it to the technology fraternity.

  • If you are a pure-play business consulting player, consider creating a business analytics offering positioned as "using information to enhance business outcomes". Side-step IT services players as providers of plumbing and foundation technology.



Posted October 5, 2010 7:11 PM
Permalink | 6 Comments |

 

Analytics is the application of mathematical modeling & optimization methods coupled with appropriate visualization, to enterprise and extra-prise information, leading to behavior change amongst business users and consequently, enhanced business outcomes for the enterprise. The early success in analytics has been seen in areas such as marketing optimization (better targeting of direct mailers or e-mails), attrition analysis in Telecom and markdown optimization in Retail.

I submit most large System Integrators (SIs) do not “get” analytics.

One definition of strategy is the positioning one creates by solving a fundamental human problem. I do not see the seminal thinking, building of foundations and the fundamental shifts being attempted by SIs to address the challenge of creating competitive advantage from information. What most SIs do instead is to model data, move data, massage data and report data.  They have built up large revenues and head-count focused on ETL, building/ maintaining datawarehouses, and building/maintaining reports. With scant regards to behavior change caused and business outcomes occasioned. SIs rarely look “inside” data.

The KPO units have done better. They look “inside” data. They house mathematical modeling skills; they engage with business stakeholders at the client; they speak in terms of business KPIs. For various reasons, they often get clubbed under BPO – even though their culture and brief is different. I submit even the KPOs have not moved the needle enough. Often they focus on scaling, automating and cost-effectively executing client-established analytical functions.

 
Why do SIs not get “analytics”?

Vision and leadership - The role models in analytics – such as Marriott, CapitalOne, JCPenney, Amazon, Netflix  and Harrah’s  – have all had visionary leaders – who saw the potential of analytics to create a competitive advantage. They clearly the set the agenda and vision for the organization. Programs were carved out, organization structures were refocused, and priorities were indicated. The discipline of collecting and housing clean data was established in a sound way.  Board level commitments were made for business results.

Why should it be any different for SIs? If you need to service an analytical enterprise, you'd better also be a believer and a champion. 


Mental model - I got to learn that, at a large SI, the proposal for acquisition of an analytics start-up was shot down because the decision makers did not see it as “IT” work. Our mental model seems to be constraining what and how we service our clients. This was tested in early 2000s when business consulting came into SIs. We suddenly saw people amongst our midst who did not necessarily have an engineering degree, did not code, did not understand web “post” and “get”. It took us several years to accept and leverage the folks with a business background. To me it looks like we are in a similar situation in analytics. We are not sure where this belongs.

Analytics start-ups that get acquired are faced with management who are unsure how to leverage them. Often they are construed as KPO and merged into the BPO business – something I have a problem with.

To compound the problem, analytics is the knotty area that requires collaboration between 4 orthogonal disciplines – business, math, visualization and IT.   If any one of the above 4 attempts analytics by themselves, the results are far short of the full potential – “using information to enhance business outcomes”.


Branding and gumption - Few SIs have the branding, gumption and the inclination to engage on business outcomes. Even during conversations around ERP or CRM implementations (and I hope one implements ERP to further business outcomes!) SIs comfortably lapse to a conversation around budgets, project schedules and project success as opposed to business measures. Analytics without a conversation on business KPIs lapses into outsourcing of data preparation, cleansing, SAS or SPSS programming and reporting. Little surprise that one of the first to set up a Social Intelligence practice is McKinsey.

Measurement and organization structure - I do not pretend to have cracked this one. But the issue here is very simple. If you measure an early stage practice or idea the same way you measure an established practice or unit (say, your BFSI unit), you have setup the early stage idea for an uphill climb, and for possible early failure. If you have an “analytics” unit (and I am not saying that it is the best way to organize analytics in an SI), the revenues from the unit are likely a small fraction of the total.  One way to solve it is to associate this with a “mass” offering, such as business intelligence and datawarehousing. Another is to conceive of other measures – such as “influence revenues” or large deals occasioned.  Quite another is change the structure to embed analytics into each industry vertical unit – a move which probably should be examined 3-4 years down the line when analytics is more mature (remember the days when “Java” or “.net” was a practice on its own?). Either way, there is no substitute to vision and sponsorship from leadership – if the chief does not believe in the analytical enterprise, you cannot go very far.

 

In summary – clearly start-ups have been successful in creating viable revenues streams around analytics. We now have got to a point where start-ups cannot flourish further without a larger eco-system. Larger organizations are trying to create analytics practices. Their success is limited because I feel they have not understood the challenge at hand. In order to be successful they need to address the issues of

  • Vision and leadership – that understands how they can service or enable an analytical enterprise
  • Mental model – that breaks away from the current business model – and that can accommodate the 4 areas needed to address analytics
  • Gumption – to engage with clients on business outcomes
  • Measurement and organization structures – to house analytics and acknowledge that it can have an impact disproportional to the revenues it brings.

Posted August 6, 2010 8:09 AM
Permalink | 10 Comments |

We solved the problem of storing and accessing large amounts of data - traditional database vendors upgraded their offerings to work with terabyte size data. And we saw the advent of datawarehousing appliances such as Teradata that could store and access data in hundreds of TB with relative ease.

While we solved the problem of storing and accessing such data volumes, processing it and looking for patterns continues to remain a challenge. For example, a large bank in the US took upto 14 hours to run a SAS AML algorithm inspite of having Teradata. If we had to look for a pattern such "A calls B and hangs up, and immediate after that B calls A" through a months data of calls, it could take hours if not days.

Why? We still need to access the data, pull it into one or more servers away from the datawarehouse to perform the computation or search.

The same process at the bank above took 3 minutes when the SAS APIs in burnt into Teradata were used.

In this case, the data did not have to be pulled away and examined. All of the work happened close to the place where data was stored. The appropriate APIs (in this case SAS APIs) resided in the core of each Teradata Snippet Processing Unit, making it 300 times faster.

Greenplum and Asterdata are two other datawarehousing appliances that have impressed me with grounds up implementations of "taking processing closer to data".  It is worthwhile giving a good look at these products - both have unique features - and much better price performance.

To take advantage of these technologies think through your processing needs as well as your data model and data volumes. 

 


Posted June 9, 2010 6:05 AM
Permalink | 1 Comment |

Who is a Business Analyst?

The traditional definitions:

  • Helps define functional requirements - what should this transactional system perform?
  • Works with business processes, performs process modeling and models use cases
  • Understands the business and helps bridge the gap between business users and those who develop transactional systems

In contrast, who is a BI Business Analyst?

In my opinion, a BI Business Analyst needs to be viewed differently.

  • Helps define information requirements. Helps envision KPIs, metrics, dashboards and other methods that help measure, visualize and manage business outcomes
  • Helps define analytical data models - those data models that help in visualizing information and decision making
  • Builds consensus between the ideal information requirements and available information - on aspects of availability, latency, quality and degree of integration.
  • Is well informed about viewing and visualizing information in a way to create value and insights to business users
  • Understands business measures, and bridges the gap between business users and those who develop BI applilcations

 


Posted April 2, 2010 5:37 AM
Permalink | 3 Comments |

 

Sure you have spent several years on developing your datawarehouse, data models, ETLs, and reports. I understand it is hard work. Sure your programs are regarded as IT successes.

But are they also regarded as business successes?

I have been speaking to many IT and business leaders anchoring large BI and analytics programs. Most seem to get this immediately.

Have you measured the business value of your BI/Analytics program?

You offer to measure and help improve business performance (that is my definition of BI). However... 

Have you done BI on your BI?

 

Let us get more specific.

  • Which reports were accessed, how often? By whom?
  • What do business users actually look for when they read each of your reports?
  • What conclusions and insights have they gotten from each report? 
  • Do they continue to glean such conclusions repeatedly over time?
  • What action have they taken after viewing your reports?
  • What performance levers /casual factors did they discover when using your analytics?
  • What impact did those actions have on specific business KPIs (WIP inventory, markdowns, backhauls, churn, CSAT are some)
  • What behavior change did your reports occasion? (for example, do despatchers now routinely look for backhaul optimization?)

 

These are hard questions. And often so enormous that we are afraid to confront them. We prefer the comfort of technology - it seems more tractable.

 

However, when we choose to confront this Gordian Knot, the answers can be pleasant. We can create BI and analytics programs that deliver more business value, at a lesser cost, in shorter time frames.

 

Here is some guidance I can provide from my experience:

  • Begin with the resolve to address 2-3 specific business pain points
  • Obtain full buy-in of business stakeholders to address the aforesaid pain points
  • Start with a sub-set of data available that is trusted by business
  • Start with a small set of dashboards that capture metrics and KPIs that link to the business pains
  • Link every report or dashboard to possible insights and actions
  • Provide ways to explore data. If a report cannot lead to insights, question its value
  • Do not look to bring all data into the datawarehouse. It is OK to use SOA - Data as a service to perform some parts of exploration
  • Seek to understand the conclusions and insights business users are looking for - and seek to automate the discovery of those insights
  • Business never worries about the norm, but looks for exceptions. Seek to understand the exceptions business is looking for and figure out ways to alert them of the same
  • Bring close-looped-ness into BI - your BI program needs to learn with every iteration
  • Explore ways to loop-back conclusions of BI into transactional systems - "automation of action on insights!"

 

Cheers!

 

 

 


Posted February 7, 2010 1:59 AM
Permalink | 3 Comments |

My hypothesis is that far too many challenges in Business Intelligence are construed as technology issues. And hence very often BI programs are driven as technology initiatives - and often fail to make business impact.

Is Data Quality a Technology Issue?

Ing Vysya bank in India created a new product and asked for a set of potential customers. IT came up with a list of 5,000 target customers - a number that shocked the business - the actual number was expected to be 10 times more.

The CIO, CVG Prasad investigated and discovered that errors in date of birth entries made the list inaccurate. A drive commenced to address this problem - both corrective and preventive action was taken. As expected, the actual numbers were much higher.

This data quality issue had the potential to tilt the balance. At a target number of 5,000, the product was unviable; at a target number of around 8 times more, the product was definitely a go.

You see what I am saying? Many times business has low confidence in the quality of data (completeness, fidelity, accuracy). This implies that:

  1. the decisions taken using this data could be wrong 
  2. the data is not used for decision making - executives instead lapse back to relying on their intuition.

Both of the above are sub-optimal.

Viewing Data Quality as a business issue helps us drive corrective and preventive action to focus on business decision making.

 


Posted January 24, 2010 8:18 AM
Permalink | No Comments |

Too many BI professionals have a self-imposed limitation that their role is to generate reports requested by line managers – guys who manage operations and are responsible for business outcomes. Consider a paradigm shift – “BI’s role is to help measure & enhance business performance”. For example, you help the chief risk officer to heat-map risk and discover corrective action; help the A/R to discover patterns of debtors, and focus collection activity.  We suggest the following practices to create next generation BI –

·         Identify specific business pain points for a stakeholder, and focus BI activity to address that pain. (e.g. “quality issues in my tire assembly seem to have trended up significantly during the last 6 months”, “there seems  to be wide variability around the lead times for delivery of my trucks”)


·         Nobody really reads a long report line by line.  A skilled business user always scans the report for patterns or exceptions – some worthy of future action, some of immediate action. Always question the purpose of long reports – “what are you looking for? Can we do that for you? ”


·         Use techniques like heat maps and traffic lighting, score cards – they help summarize the information and help highlight areas for possible action.


·         Create early POCs and demos. Show and tell. Shh…. you know what, business users do not know the difference between the mock-up and the real thing.  So they feel extremely excited when they see an HTML mock-up; they feel much less excited with a Word mock-up or a PowerPoint.  We created screen mock-ups for critical processes for the Consumer Goods industry; we got rave reviews and a lot of interest by using the POC for a show and tell.


·         Help identify exceptions for action. This may need additional data not contained in the datawarehouse – do not let that constrain you.  Currently this is performed by humans looking at reports; there is no reason why software cannot be trained to do this.


·         Have a layer of SMEs  who could perform the first layer of interpretation – so that business can focus on the next layer & actioning.

Of course, the foundation datawarehouse needs to be available to accomplish all of the above.  Cleanliness of data and its integrity is key to make the above successful. Does this mean that the BI department is no longer a technology group that only creates and maintains reports? Yes. This evolution is long overdue.

Do business stakeholders want this? If you ask, they will say “probably” or “no”. Business cannot want something that they have not experienced. Try a POC. Show them and they will come running.

Can this be outsourced? Hmmm…..

As someone who has spent more time in the outsourcing industry than on the other side, my view is as follows:  This is a very high value-add function. Many outsourcing companies are architected for volume – repeatedly doing an established process at a lower price. My personal recommendation is to program manage the POC till you demonstrate the value addition. Atleast ensure that >50% of the job has becomes codified. Then consider outsourcing the important function of information interpretation.

Look out for my subsequent blog postings: 

·         How do we design the organization for information interpretation? 

·         Can we get software to perform the first layer of report interpretation?

·         How do we leverage outsourcing companies in BI?


Posted November 24, 2009 8:21 PM
Permalink | No Comments |

Business intelligence programs could be successful IT endeavors but often fall far short of making the desired business impact. In other words, DW & BI project teams working on say Credit Risk analytics have delivered their models and reports, but business is still left with the challenge of gleaning insights & trends worthy of action. Most businesses get so frustrated with the state of affairs that they often assemble their own "BI" teams to obtain these insights and actionables.

In this first part of a multi-part blog posting, I offer analyses, along with simple and practical methods for BI programs to become shop-usable. These tips will be useful for core business functions (CEO’s office, Risk, Compliance, Supply Chain etc) as well as the IT function.  I willl also call out  ideas for services organizations (SIs) to craft a next generation BI services organization.

Let us address the requirements gathering for BI. IT often likes to think of BI requirements in terms of "tell me what reports or dashboards you like to see. Tell me what data items and dimensions  you need, and the report format".   IT tends to dumb requirements to a point where it becomes an IT convenience.  A transformational question is "tell me how you plan to read these reports, what trends & insights you could possibly glean, and what decisions you expect to make?" In other words, it is important capture the unarticulated framework being used by the business stakeholder to make determinations and commence action. Providing recommendations (even if parameterized)  gives an action focus to BI. Whenever we are creating  reports in hundreds for the same or similar stakeholders, be sure much of it is not used, and not considered valuable. (report portfolio optimization – a subject of my future blog posting).

If your BI program is not providing recommendations for action (because you presume it is "their" job), be sure the usefulness of your reports is limited – essentially you are saying – "this is all the data guys, figure it out by yourself".   Re-orienting the conversations with business stakeholders around measures, insight and recommendations has the potential of  IT to be seen as a positive business partner as opposed to a data provider.

In subsequent blog postings, I will give examples of the framework to be used, the roles needed to make this happen.


Posted November 15, 2009 7:03 AM
Permalink | 8 Comments |