Mark Cunningham has reunited some of the team that built Crystal Reports (now part of SAP Business Objects) and launched Indicee, a SaaS-based BI reporting play that is pointed squarely at the continuing difficulty of extending BI beyond its seemingly permanent minority usage model.

It's commonly understood that users continue to fend for themselves manually, moving data to spreadsheets for analytic manipulation because IT is unable to respond quickly enough to their needs. Indicee tackles this by re-using existing report and spreadsheet content (not surprisingly, Crystal reports lead the source list), moving it to the cloud for data mart-based interaction, and innovating a different approach to user interaction. It's worth a look, and a free download for trial use sweetens the deal.

more...

IBM chose a regular meeting of its Data Governance Council to launch a series of now-renamed Information Governance services and products and announce a definitive agreement to acquire Initiate Systems. A privately held software vendor based in Chicago, Illinois, Initiate stands as IBM's 30th acquisition in the information and analytics sector. Initiate's data integrity software for information sharing is designed to help health care clients work more intelligently and efficiently by supporting timely access to patient and clinical data. It also enables governments to share client information across multiple agencies to better serve citizens.

This post was co-authored by Merv Adrian and Charles King of PUND-IT. It's a little more narrative than my usual posts (and longer), and is reproduced here in the form it appeared in the PUND-IT newsletter. 

more

This is a new kind of blog post for me - it's specifically designed to elicit ideas, suggestions and examples for my Big Data Summit (click to register) kickoff on Feb 18th. Consider it an experiment in crowdsourcing from a very smart community (my readers.) No, I'm not a shameless suckup - you know who you are, and you're all pretty darn smart.

We have some great speakers - Amazon, Mobclix and Aster Data - and added sponsors like Informatica, Dell and impetus. So there will be plenty to talk about, including storage, analytics, programming paradigms, etc.

Here's my dilemma: I imagine we'll have newbies as well as more advanced attendees. So I cannot skip the basics like:

• Store it yourself? Host it elsewhere? Move it and process, then return it?
• DB or not DB? (subcategory: SQL or NoSQL?) and, a corollary:
• Where should analytics be done? Inside or outside a DBMS?
• Where are the resources to learn more?
• Who are the providers of key technologies?

This may take my 30 minutes and I'd hardly scratch the surface. But maybe I can blast through it fast and get to some other issues. What would you want to hear if you were there? Or if you're involved in projects and have learned some lessons you think the world should know, what would they be? I'd be grateful for any input my readers have to offer. Feel free to send me links, experiences, interesting use cases you've seen or participated in. Put them in the comments below.

Informatica has announced another, long-rumored acquisition: Siperian, thus continuing a steady march toward a comprehensive portfolio play. In 2009, its strong growth path made it the clear independent leader in data integration.  With Release 9, its vision of a data integration platform grew to providing a comprehensive approach to everything from data discovery services to data quality. While growth slowed during a tough year for the economy overall, Informatica grew revenue in every quarter, and made key acquisitions in 3 successive quarters (Applimation, AddressDoctor and AgentLogic) and began to make significant moves into the cloud via partnerships with Amazon, Salesforce and others. Agent Logic added event detection and processing to support real-time alerting and response. As 2010 begins, this latest move is synergistic from the outset; Rob Karel points out in his excellent blog post that "Siperian MDM technology...already is deeply integrated with Informatica's identity resolution and postal address technology. In addition...Siperian MDM customers [are] using Informatica for data integration and data quality, meaning there is a lot of existing experience and know-how on integrating Informatica's portfolio with Siperian."

 

more

After 27 years of steady growth, Austin, Texas-based Pervasive (PVSW) has become a $47M annual run rate software provider. Its portfolio includes a "zero admin, light footprint database" (the former BTrieve, now PervasiveSQL), data integration software (for SaaS and on premises applications), and data synchronization products for such apps as salesforce.com, Quickbooks and Microsoft Dynamics CRM. In 2009, it began leveraging its DataRush processing engine as a product, providing a solution for companies that want to take advantage of multicore architectures to drive dramatically enhanced performance on much smaller footprints, for programming data services tasks such as aggregation, de-duplication, cleansing, integration, matching and sorting, as well as data mining and predictive analytics.

More

ADBMS vendor SAND Technology's report on its 2009 fiscal year seemed to offer little reason to change my earlier skeptical position on the firm. Its 2009 revenue was essentially flat at $7 million (Canadian dollars throughout). Cost of sales, R&D, and SG&A - and the firm's net loss - were also nearly unchanged. And yet, there are changes going on, and they are positive signs, especially for a year in which the IT market will rebound. Net income for SAND's fiscal 2010 first quarter was $553,253 on revenues of $2,485,464 - a substantial turnaround from a net loss of $989,850 on revenues of $1,223,928 for fiscal Q1 2009. One quarter is not a trend, but it is a good sign.

For more, click here.

Disclosures: SAND Technology is not a client.

(Co-authored with Charles King of PUND-IT, Inc.)

Microsoft and HP announced a new investment of $250M into their Frontline Partnership, designed to deliver integrated stacks supporting applications from Microsoft's Exchange and SQL Server and beyond into the cloud. As part of this effort, the companies plan to deliver solutions built on what they defined as a "next generation infrastructure-to-application model" which will help speed implementation, eliminate IT management complexities and lower overall costs by automating manual processes. With this strategic partnership, HP and Microsoft will also collaborate on an engineering road map for joint products including data management machines using the new SQL Server MPP database option when it is announced, pre-packaged application solution bundles, comprehensive virtualization offerings and integrated management tools.

To achieve these ends, the pair intends to collaborate in the design of pre-optimized solutions for Microsoft's SQL Server and Exchange, virtualization and datacenter management. They will also develop SMB Smart Bundles which will be sold through their 32,000 common Frontline channel partners. In addition, Microsoft and HP will promote these mutual efforts by citing one another as "preferred partners" in virtualization and datacenter/IT infrastructure management solutions and scenarios. As a result, Microsoft and HP said their customers will enjoy increased business efficiency, improved application performance, enhanced operations and investment protection.

The Pitch

Microsoft and HP hope to find solutions, sales and safety via collaborative efforts and mutual self-promotion.

Mission Accomplished?

Readers should be forgiven if Microsoft and HP's new announcement elicits a sense of déjà vu. After all, the two companies have worked closely for years, share tens of thousands of common customers and channel partners, and have long supported one another's best interests. The partnership has been in place for over 20 years and has been well regarded. But this latest announcement suggests that the pair's already considerable collaboration is deepening monetarily via their combined $250 million investment, technologically through the code-level system integration/optimization they plan and strategically with their new, mutual preferred partner status.

How important or notable are these elements? That varies. While a quarter of a billion dollars spent over three years is hardly chump change, it does qualify as small potatoes for a pair of companies whose combined annual revenues are well north of $150 billion. The technological collaboration is more interesting, since it will occur at a fundamental code-level. The real question--and one that will not have concrete answers for some time to come--is just what sort of granular system benefits will result from those efforts.

Equally intriguing to us is the pair's plan to confer preferred partner status on one another. In a way, this move was highly predictable. For well over a year, vendors that own an entire datacenter stack of hardware, middleware, virtualization and management technologies (i.e. IBM and, more recently, Oracle/Sun) have begun focusing on developing datacenter solutions optimized for specific applications and workloads.

More recently, creative collaborations including Cisco, EMC and VMware's VCE Coalition demonstrated how vendors with more specialized offerings could effectively play the same game. The new Microsoft/HP new partnership fits firmly in this latter category, and provides some competitive remediation for both companies: HP's software stack is riddled with holes when compared to IBM's and Oracle's; Microsoft has no enterprise hardware play at all.

But while the effort could well provide both companies new benefits and opportunities, it is also likely to result in significant challenges, especially concerning their other critical partners. How will Dell, IBM, etc. react to Microsoft's declared preference for HP? Should VMware go out of its way to support HP when the company is promoting Microsoft's Hyper-V virtualization platform? Just as important, how will the thousands of Microsoft and HP channel partners who earn their daily bread by integrating and optimizing SQL Server, Exchange and other Microsoft solutions feel about those processes being incorporated into manufacturing?

In the SMB market, the value-add is often the degree to which those partners handle installation and integration. Today's message seemed to imply that much of this business might go to HP's services organization in the more simplified world of fully integrated SKUs offered as a single purchase. The specific focus of the announcement on Exchange and SQL Server deployments strikes directly at the heart of two of the most lucrative solutions offered by Microsoft's channel partners, and suggests that many may find themselves increasingly competing with HP.

Certainly the two vendors' partners have a formidable array of resources at their disposal. Microsoft's Worldwide Partner Group VP Allison Watson was already prepared with a crisp, actionable message. Her pitch: As a result of the deepened alliance with HP, Microsoft partners will be able to deliver solutions faster and more easily. Can they improve their margins in the process? That will depend on what HP will charge for similar implementations, one assumes.

Watson's message points to tools for easily sizing systems running SQL Server either for transaction processing or for business intelligence. It has also offers links to HP Insight orchestration templates for HP's BladeSystem Matrix and white papers. Assets in place like Microsoft Sales University will accelerate partners' ability to get tools and training--free--that ensure their ramp is as fast as possible. Still, HP's teams in the field will likely have the inside track, so the underlying message is clear: move fast, or be left behind.

Overall, we consider Microsoft and HP's new partnership an intriguing collaboration that could broaden and deepen the two companies' ongoing efforts. If it performs as advertised, it could also deliver significant dividends to the pair's business customers. Certainly it provides a partial competitive answer to the full-stack stories of their large competitors and other collaborative partnerships. But we remain uncertain and wary of how so exclusive a relationship will sit with and play among the industry and channel partners critical to Microsoft and HP's overall success.

As for delivering the new "applications" model, this announcement fell far short. The only applications mentioned were Microsoft's SQL Server, Exchange and System Center (the latter of which will be integrated with HP Insight), already focal points for the Frontline Partnership. Microsoft's business applications for CRM, ERP, etc were not mentioned. And building credibility for pre-integrated infrastructure solutions, even for relatively simple SMB deployments, will require discussion of other applications from other vendors. As interesting and promising as Microsoft and HP's announcement was its scope is far narrower than the companies' "Next Generation Application to Infrastructure" messages imply.

© 2010 PUND-IT, Inc. and IT Market Strategy. All Rights Reserved

Sybase has quietly racked up a string of successful growth years, riding its pioneering status in commercial analytic databases (ADBMS) and holding on to its loyal base in everyday DBMS after being elbowed aside by Oracle a decade ago. Its steady market performance has not been driven by dramatic innovations: Sybase has seemed to lag the Big Three (Oracle, Microsoft and IBM) in new feature/function. But it has innovated: IQ has grown into a key revenue source, and Sybase RAP has established itself as one of the more succesful event processing offerings, with a string of Wall Street customers creating a new class of applications.

In the current (5-year-old) major release level of its flagship Adaptive Server Enterprise (ASE) product, Sybase has added user-defined SQL functions, support for plugin Java Runtime Environment (JRE) and JVM components, xml tables, SQL statement replication, new statistical aggregate functions, and a shared disk cluster edition. And now, Sybase is about to add new in-memory database capabilities and step up its support for external storage management. I've spent some time recently with the Sybase team to discuss their plans for the upcoming 15.5 release (currently available in a developer version), and found palpable excitement about the possibilities of their new work.

One of the most obvious opportunities is to think about the role of I/O and disk storage in performance. Much of a database's code line is spent logging, writing to disk, managing buffers, optimizing access strategies, and ensuring ACID properties for transactions. ACID properties (Atomicity, Consistency, Isolation and Durability) have been the hallmark of DBMS thinking since the earliest days of computing for transaction processing. But the assumptions they were designed for are not universally necessary - not all actions taken with data are transactional, and persistence may already be assured for non-transactional data elsewhere. Moreover, computing architectures have changed dramatically since the earliest days of transactional theory, and much of the thinking was designed for worst case scenarios occurring in system designs constrained by technology and economics that have since changed.

In-memory databases have been shown to create a substantial uplift in performance, but adoption has lagged. The shift in processor architectures to 64-bit is hardly new; decades have passed since MIPS and DEC introduced 64-bit microprocessors into mainstream computing, followed by Sun, IBM and HP. But software architectures that leveraged the opportunity lagged; operating systems took a long time to exploit the possibilities of a theoretical address space measured in terabytes. Even with the limitations of today's systems and the cost of memory (which is continuing to fall rapidly), this is exciting stuff. But it's been over a decade since Solaris 7 arrived in 1998, following on DEC's Alpha-based OSs. And while Unix and Linux variants running on 64-bit architectures arrived, databases continued to be slow to re-architect for the new paradigm: DBMS vendors ported their products to run there, but didn't dramatically re-engineer the parts of the codeline that could profitably take advantage of the new environment.

Many database products have complex features and options that can (in fact, must) be managed to achieve optimal performance, by relaxing assumptions and changing processing defaults where it is possible to do so. But the arcana associated with this has been the province of specialists - like those who tune the systems used for TPC benchmarks, or the top-notch staff at financial institutions responsible for trading systems. Sybase is exploiting the opportunities in a different way, by creating (rather than acquiring) a product whose codeline has been explicitly designed for in-memory use, and configured for two specific use cases where it is appropriate: one for data that does not need to be persisted at all ( a completely in-memory, no-disk instance) and one with "relaxed durability." In this way, Sybase expects to dramatically expand the number of customers who can take advantage of the high-performance opportunity - and reach a new group of customers who may not have been considering ASE for many of their new projects.

Sybase has taken a different path here: instead of acquiring in-memory technology, like IBM's SOLID DB or Oracle's TimesTen, it has built an in-memory version of its existing code, reductively, ensuring compatibility and ease of interoperation much more rapidly than its competitors. This will make it far easier for its customers to build applications that leverage multiple persistence profiles in different database instances. In an age where many applications depend on look-aside processing, compliance management, and similar out-of-transactional band activities, this could be a powerful new attractor for modern workloads. Databases that are kept in memory for lookup purposes can be recreated automatically at startup from "templates," so assumptions about their recovery can be completely different. Database elements like tables, stored procedures, datatypes, etc., are included; Sybase's TSQL and all APIs are supported immediately. Management, monitoring, and the like will also be seamless from the product's release, because no retrofitting will be required and no interim skill acquisition will be required for customers.

Another major step forward in systems architectures has been the growth of smart storage and associated management software, a requisite in a world where data volumes continue to explode and managing data is a task that goes on outside, as well as inside, the DBMS. Sybase's customers, like those of its competitors, have heterogeneous environments with multiple database products and storage options. Recognizing that external storage management is frequently chosen, Sybase has partnered with IBM to leverage Tivoli Storage Manager (TSM)'s  ability to work with multiple DBMSs and store across multiple types of disk and tape storage. Sybase's Backup Server will integrate with TSM to hand this off.

These are solid steps forward, and should allow Sybase to continue to move forward on its successful growth path. While they are neither dramatic enough nor likely to be marketed aggressively enough to push Sybase into the top revenue tier, they are well-designed to allow the company's steady growth to continue and even accelerate. Sybase has picked up market share in the past few years, but larger changes are afoot as the rise of open source, cloud computing, appliances, and proliferating analytics workloads change business and deployment models in the decade ahead. And Sybase's "Unwired Enterprise" positioning around mobility, as powerful as it has been, hasn't been connected to the ASE story here. New strategies will be needed from all the vendors as they confront the industry transitions ahead. Dramatic innovations at the edge will continue to nip at the heels of the largest players, picking off carefully targeted use cases. I'll talk about many of them in the months ahead.

Since my piece on Oracle's recent TPC-C was posted, interesting emails have pointed me to additional price/performance data, and I thought I'd offer a bit of that to my readers. One of the more interesting came from the admittedly biased Conor O'Mahony, a DB2 product manager for IBM. In his blog, Conor points out some interesting elements to Oracle's pricing and support for the system tested. To wit: "the IBM result includes pricing for 24 x 7 support, upgrade protection, and perpetual licenses; the Oracle result does not include any of these features." It turns out that Oracle uses a less costly, 3 year term license for the benchmark. After 3 years, the user has to re-up (or just buy a regular license.) The support piece is equally interesting; Oracle's Incident Support offering - with up to 10 Web-based incident requests per server and no phone support or future upgrades - is used for the benchmark system pricing.

Benchmark critics like to point out that vendors use a lot of "benchmark specials." Technical ones includes features being turned off, and special code inside the DBMS to optimize known tasks the benchmarks test. In both cases, understanding the tweaks is useful, because you may in fact be able to apply some of the "tricks of the trade" if your workload and policies permit. I've always believed that optimizing some of those known tasks did in fact advance the state of the art substantially over the early years of the RDBMS. But games with licensing and support are different - it's rather unlikely that purchasers of systems that depend on high-level performance would make the choices above for production systems.

O'Mahony has another post that looks at a recent SAP benchmark with some similar issues to those I mentioned in my post about the TPC-C, with the addition of some "undocumented, unsupported features" that were employed. If SAP performance is one of your areas of interest you may want to check that post out as well.

I recognize that this post, and my earlier one, appear one-sided to IBM's advantage. That is not my intent, and I welcome substantive comments delving further into the issues. My strongly held belief is that Oracle makes a good product; they are not the leader for nothing. And they have brought innovations to market ahead of their competitors in clustering and memory usage, among others. They deserve credit for that, and the benchmark results speak for themselves. That's why I dislike seeing their value obscured by other issues like the ones described in these two posts.

Rod Adkins, the SVP and Group Executive of IBM's Systems and Technology Group (STG) took the time to engage the influencer community quite early in his tenure for a well-run event at the Watson Research Lab in Yorktown Heights. "I've been in this position for 38 days," he  reminded us, as STG's AR team widened the usually hardware-focused invited audience to include generalists and more software-focused folk like me.  IBM execs from IBM's Software Group, its Research organization and corporate, joined us  for a look at the science behind the systems, a compelling addition to the agenda. And another pitch for IBM's analytics thrust was a scene-stealer.

Adkins is hardly new to the party: he spent 10 of the last 12 years in STG in addition to the work on pervasive computing he did in the Software Group. "We play a vital role in the Smarter Planet initiative - we have to provide optimized infrastructure to capture, manage and deliver all the data it runs on," he pointed out. He clearly articulated a strategic path for 2010 to match his aggressive influencer communications agenda: growth in workload optimized platforms, systems software value capture, delivery model changes and data center wins. For each, organizational changes, investments, messages and new offerings are well scoped and already underway. STG has 47% of IBM's R&D budget and is investing in process and packaging; technology design; hardware systems; systems software; and client support. Yes, research into client support - a topic well worth its own discussion, but beyond our scope for this post.

Adkins takes the helm of a ship in good order: 2009 performance to date has been steady, and compares well with its peers given the economic environment. Revenues for Q3 were down 12 percent from the third quarter of 2008 (better than HP's 17% decline in its server group). IBM's mainframe, late in the z10 product cycle, decreased 26 percent. It's not unusual to see declines at this stage as the market anticipates the next (usually well known) version. Still, z continues to defy all predictions of its demise: it has almost doubled its share of systems over $250K in the last 8 years from 17.2% (Q400) to 32.1%(Q408). Adkins touted IDC and Gartner reports of market share gains for Power Systems, System x and disk and tape storage during the third quarter. According to industry analyst sources (IDC and Gartner) revenues from System x servers increased 1 percent; microelectronics OEM revenues were fairly flat at a 1 percent decline. The 45 nanometer business is running ahead of the previous (65 nm) ramp - its production run is already sold out and fab performance is good. IBM's acquired XIV storage has shipped its 1000^th unit - 70% of which went to new customers. And IBM claims to have capitalized very effectively on the turmoil in Sun's base: "We've won business from 100 of Sun's top 300 customers."  Even system software has a few things to crow about: the Migration Factory solution, IBM asserts, has driven 2000 migrations to IBM POWER systems so far this year. 

IBM is very optimistic about upcoming technology introductions: its next generation Power7 chip architecture will feature substantial performance and energy consumption improvements, as well as some interesting new dynamic thread management capabilities. Big bets are being placed around the notion of "workload-optimized systems," pre-integrated solutions such as:

• Cloudburst  private cloud hardware appliances for secure deployment and management of  application environments
• The new scale-out NAS (SONAS), for standards-based, large scale file storage
• Smart Analytics System,  a pre-integrated  analytic appliance that delivers IBM data warehouse software preinstalled on an IBM server and storage

Some innovations are delivered as what IBM is calling "system optimizers." For example, the Smart Analytics brand is extended to the Smart Analytics Optimizer, a software-based accelerator (available now for z and moving to other platforms) that automates the movement of frequently queried data into as much as a terabyte of main memory, where other tricks like vector processing and a new "optimized storage format" (I haven't explored edetails of his yet) IBM claims will dramatically improve performance. IBM's decades of database  optimizer research enable it to drop this in and "know" when to push queries to the in-memory database. This kind of hybrid system will change the architecture of data management if it is proved to work as it rolls out - and I wouldn't bet against it.

IBM has been ramping its geographic coverage with 56 added global branch offices and will add 15 more in 2010. The channel is getting more investment in education, enablement, and training in how to improve profitability. And STG has created a new System Software business unit to drive development, revenue and margin growth, with a service management focus that will take HP on in one of its core areas of strength.

Where STG Fits in the Smarter Planet Story

John Iwata, IBM's Senior Vice President Marketing Communications, gave us a conversational look at how IBM continues to apply deep probes into its own business to understand company customers, their needs, and resulting opportunities. "We looked at 20 showcase IBM solutions that we had built and implemented for clients," he said. "We found that all 20 had the entire portfolio of IBM product and service, technology and in almost all of them the Research division played an important role in the eyes of the client as differentiation as to why they chose IBM, and" - perhaps most interesting - "are  instrumenting their physical systems. By 2010 there will be 1B transistors per human; there has been a hockey stick in deployment. The internet of smart things, and the associated data, are the wellspring of IBM's Smarter Planet thinking."

By now, some of the examples are familiar but Iwata unveiled one I hadn't seen before: smart sewers. Who would have thought? But IBM predicts at least $70B of expected investment in sewer modernization. In San Francisco, smart manholes have a float attached to a wireless transmitter to gauge water flow and water level.  Our sewer systems, many of which are a century old, are built for average volume, but when you have above average flow they are easily overwhelmed. It's a familiar phenomenon. And worse things happen: as I write this, a 100-year old main is being repaired after rupturing two days ago and causing massive problems, creating a giant sinkhole flooding streets and buildings, and closing street. Perhaps future systems will be able to anticipate these failures better.  Iwata pointed out that other instrumentation might let you learn that a certain pump needs to be fixed; doing so beforehand avoids similar failures and catastrophes. And, of course, cities will get smarter than "managing by averages" permits by tracking in real time, and learning from the data.

Iwata noted that "We don't label anything 'dumb' in our messaging. Nobody designed the systems that way." Advances in technology permit new ways of thinking, even though, as he acknowledged, the timing could not have been worse in terms of the economic conditions. It's a leadership agenda, not just an ad campaign. IBM took Smarter Planet to governments all over the world - it can raise job growth. Much economic stimulus money flows here.  Iwata has been delighted to see how the field has taken this program up enthusiastically - they don't always, do so, he acknowledged wryly. Over 80 field-driven events have been done, with Ho Chi Minh City being one of the most recent, focusing on a "smart food" supply initiative. Emerging markets are looking to leapfrog from traditional systems to post-industrial, information age ones. IM stands to gain enormous opportunities from all this. It can do well by doing good.

The Mathematician's View of Advanced Analytics

The star turn from the wings at this event came from Brenda Dietrich, a last minute substitute when a snowstorm in Minnesota prevented a Mayo Clinic presenter from joining the meeting. Dietrich, an IBM Fellow, and VP from Research, has made many contributions over an illustrious career; she shared with us that she coded the 1959 algorithm underlying GPS as a new intern and was the largest user of computing power at IBM that year. Today's GPS device makes for a good example of the potential for advanced analytics, when one thinks about it.

Dietrich says analytics can be applied in many places - for example, she and her team have been modeling IBM's biggest customers, and their own sales organization - and have made recommendations on making adjustments to the configuration of their sales force. Some of the changes were implemented - and they've seen large lift (over $1B.)

Dietrich pointed out that the state of the art in statistics is to "extract the explainable and extend." Of course, sometimes that's not as easy as it may sound. For example, the Super Bowl occurs on a different day every year - how to put information on its impact on your sales back in to your predictive model to leverage it in retail (to sell more big TVs, or just chips and beer) is not obvious. She also talked about the frontier cases - such as streaming data, which opens the door for a new type of statistics. Massive data sets from the mobile web, like GPS and OnStar, could be the basis for exciting new opportunities if used effectively. And her proximity to the STG teams gives her a view of how the science of analytics can benefit from new architectures. She noted that "most of the algorithms in the field assume serial, not parallel, hardware - so a lot of work on things like matrix inversion can be rethought."

Finally, as a truly provocative thought, Dietrich challenged us to consider what would happen if computers got smart enough with type checking at a higher order of computation. What if a built-in help system could tackle the garbage in, garbage out problem by telling users "I'm sorry, but there isn't enough data to make that prediction?" Think of it as "warning labels on mathematical models." Might have been nice for the quants on Wall Street to have had that - although there's no guarantee they wouldn't have ignored it, as they did with other warnings that Dietrich said ware not only visible, but had been noted.

It was easy to walk away from this presentation impressed by the inside look at how advance and predictive analytics are thought about - and many of the attendees were. Even more intriguing was the obvious connection to Smarter Planet - it's in the very nature of these techniques that they can do things with the newly instrumented physical systems that have not been thought of before. Delivering on the promise is one thing, but IBM is after something bigger: finding opportunities that have not been thought of yet. Few would have thought of mathematical work of this kind as creative endeavor, but Dietrich's enthusiasm, command and passion make it clear that it is just that, and the best is likely yet to come.