The Big Integration

I guess I should tell you right away that I got my new and first iPhone two weeks ago. Don’t worry though; I am not going to talk about it. We’re still in our honeymoon, so I couldn’t be objective. Sure, the bride already shows some drawbacks, but she’s a beauty.

I was in Quebec City last week on an engagement. I brought my wife along (the first one, that is), so she could enjoy the 400th anniversary celebrations (not ours of course, Quebec City’s). We were in a restaurant and were talking about making a reservation to another one for the next evening. Then something new in my life happened, and I shall relate it step by step:

1) Slide open the iPhone

2) Tap on Safari, and bring up Google

3) Enter the restaurant’s name and “quebec city”

4) Got this result at the top of the organic SERP (note that Google is suggesting the right spelling for “restaurant”, which I didn’t spell right in this example!):

# The beauty with the iPhone is that the phone number is clickable

5) Tap on the phone number

6) Phone rings

7) Make the reservation.

I must confess that, for a few seconds, I looked at my wife in awe (OK, she’s really is good looking); I had just done something I had never done in my life (and I suspect this is not the last time).

My phone call was a direct result of my organic search. I never clicked on the web site link though, which means that the restaurant’s web site analyst will not see that visit, not that organic search brought them business that night (he’s looking at his stats the day after this happened, remember). Well, of course, it was not really pure search, as I knew where I wanted to go, but just needed some contact info (Yellow Pages anyone?).

As analysts, how do we account for such a phenomenon? That “visit”, the actual one in the restaurant that is, left no trace on the web site, still originated from the internet. No data points to analyze; no real way to know, maybe just glad the manager renewed the listing of the number with Yellow Pages (!).

I think that in order to really account for those new consumer behavior patterns, we will need to finally start publishing web-only phone numbers. It is not expensive to do, and most carrier have usage reports for 800 lines. Sure, this will be harder for those who brand their phone number, but how important is it when it is shown in a search result, really? Unless Google finds a way to account for a click on a phone number when it’s done from a smartphone, and make it a part of your GA reports. That’d be very cool.

By the way, the whole experience at L’Initiale was memorable…

There was the news yesterday that some of the major cell phone carriers in Canada will start charging 15 cents for each SMS message customers receive. Previously, users were charged for messages they sent, not the ones they got. As marketers, we can only imagine how delighted consumers will be to be charged for the promotional messages we send them on their phones… I can only see all the opt out and hate mail coming, with soon a “Don’t Friggin Advertise on My Cell Phone” (or DFAMCP for short) list.

Wait! I’m not saying people don’t have the right to be left alone from being over marketed. But I can see here a new hurdle for mobile commerce in Canada. I mean, following those new rules, you would get charged 15 cents for an order confirmation message, for Pete’s sake!

Cell phone carriers should get smarter, and realize they could make way, way more money if they were set up to do some deep mobile analytics and sell special services to advertisers…

Anyway, I guess we are close to the days when real mobile Internet and mobile commerce are adopted in Canada, more than 10 years after countries such as Japan and Finland.

Kevin Hillstrom’s got us web analysts in his gun telescope. He is telling us how well Multichannel Forensics is a great framework to work with, especially on the predictive side, stuff we don’t do enough (and by the way where a LOT of value resides). You can read it here. BTW, you’ve read his great book, Multichannel Forenscis, right? No?? Ah man!

I am launching a new series, called “TBI Alerts”, which will quickly point you to some great posts or resources in all data integration matters (web data with enterprise data that is). OK, some of you will call it “lazy blogging”, and I guess they won’t be entirely wrong. But anyway, you already know how much I like to share.

So, the first alert, which is not exactly one since it was published two weeks ago (I just finally read it today), is a very interesting post from the always interesting Ron Shevlin at Marketing Whims. Ron talks about a study that shows data integration success is more culture than organizational chart.

Data integration is all the rage now, as you loyal readers of The Big Integration know all too well. Omniture has recently added a survey component to its suite of products, which adds now the attitudinal dimension of analytics. For a good review of Omniture’s recent additions, read Stéphane Hamel’s blog here and here. While you’re at it, download WASP if you haven’t done so yet.

I was just reading Jason Burby’s latest ClickZ column “Understanding Visitors’ Desires” about Omniture’s survey module. Jason points out interesting implications to adding attitudinal analysis to the current behavioral one; I particularly like what he says about what a potential integration of attitudinal data with testing data would bring to the table.

Allow me to quote him extensively:

Over time, I’m sure Omniture will integrate the survey tool with its testing tools, allowing companies to measure the success of A/B and multivariate tests based on not only what people are doing but also what they wanted to do and what impact it had on them from an attitudinal basis.

The best example of why this can be a problem is to think about trying to increase your conversion rate for any given behavior from 3 percent to 3.25 percent. If a test’s goal is 3.25 percent and you exceed it in tests by increasing the conversion rate to 3.3 percent, you may be thrilled and see it as a huge success. But what if that test turned off the other 96 percent of your visitors so much that they will not only avoid your site in the future but won’t consider your products and services down the road or recommend them to their friends? If you’re only looking at behavioral data, you won’t know this. Now in the real world, you most likely wouldn’t increase your conversion rate while upsetting everyone else, but the attitudinal surveys can tell you if you are frustrating even a small portion of that 96 percent.

Nicely put hey? Who knows… I don’t know if one of the test recipes/versions could have such a negative impact on visitors’ perception, but it’s possible.

Expect to whole behavioral analysis market to aggressively move towards integration this year (finally!). Once it’s done, we’ll have to ask the deep question of what is web data versus data from the web. Ah! by the way, that’ll be the topic of my next post (soon, I promise).

Excellent introduction to online-offline data integration by Avinash Kaushik today. He presents very practical ways to understand mutual impacts of the channels. Don’t miss your chance to read it here.

Of course, you are already a reader of Kevin Hillstrom MineThatData Blog. Ah! Funnily enough, Kevin too encourages you to read Avinash’s post ;-). Well, no mention of that post on Jim Novo’s Marketing Productivity Blog, but you probably know that Jim is mandatory reading.

OK, enough with promoting other people’s blogs. You’ll excuse me; I’m such a groupie.

Back in Montreal after a very intense week at the TDWI World Conference. I must confess: the high mileage took its toll on me on the last day. I was supposed to attend Technology Architecture for BI: Planning and Design of the Technical Infrastructure by the very competent Deanne Larson. It turned out to have so much content, and very technical, that I could not absorb any more network maps. I however got amazing material about the topic, and this has definitively made me understand a lot of stuff I was hearing about (data warehouse, operational databases, ETL, etc.).

In the afternoon, I decided to switch and attend John O’Brien’s Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence. The first part was again quite technical with a long presentation on how SOA (Service Oriented Architecture) will impact the field. I would be lying if I said it got me all excited, but I could see how my co-attendants got interested. The part I liked most in John’s presentation was what he calls the “Google Effect”. His point was that Google has made us all so much used to a certain type of experience (fast, easy, relevant, free) that this is slowly creating new expectations from BI users: easier and faster access to better organized content (i.e. reports) is demanded more and more. I thought this was quite clever, since Google is now an important part of most Internet users’ life, and I think we can reasonably think that the type of user experience Google offers through their applications will somehow percolate to all things IT, with users demanding better performance with much easier and intuitive experience.

I didn’t stay for the full afternoon, being less interested by the Web 2.0 part, and wondering if I could catch an earlier flight (of course, there wasn’t one), but frankly too tired to take anymore stuff. My fellow participants were right to warn me at the beginning of the conference that six full days was tough.

I got back here with SO much to study and learn. I have now a fairly good mental map of what the DW & BI space is, and I am already beginning to see how Web Analytics could fit in. It has also brought up many questions about what Web Analytics should be in the future, and what could/should be BI regarding Web data. Questions I don’t have answers to yet, but will certainly develop in the months to come.

When you are an independent consultant like me, you go to those events on your own dime: cost of registration, travel, and missed billing all impact your own pocket. I must say that the TDWI World Conference was worth every dollar.

I guess I was naive. I thought that Web analysts were the ones who had to live in an approximative world, because of the nature of our data. Boy, can data get dirty in the BI world! I attended the full-day course today on Data Conversion, Consolidation, and Cleansing – Practical Skills given by the passionate Arkady Maydanchik. Arkady can pack the most number of pieces of information in 10 seconds I have ever seen! He actually made what many would consider a rather dry topic, something exciting. I am not kidding; data cleansing is very complicated to execute well. I’m sure it can be very absorbing.

So, yes, data can get really really bad in that world, but at least, they have means to work on that and make it better. I realized today again how little control we have on our data, since we basically have to trust the vendors on that. What is a bad “record” in Web Analytics? Is getting back to analyzing IIS logs a solution? Well, I wouldn’t go that far; I am a true believer in tagging. But I wonder if we could not get rid of proprietary data format, and work as a community towards standardized structures in how the data is collected. This would mean that logs collected from tagging could be analyzed by whatever Web Analytics product you purchased (or didn’t). I guess this is similar to the meta data situation with BI application vendors. It was clear yesterday at the night course that none of them wanted to make their meta data readable by other vendors. I don’t know anything about that field, but it seems that meta data schemes are pretty much what they base in part their competitiveness on.

I am not sure that a call for standardized log structure (in tagging I mean; I know IIS or Apache are already vendor-neutral logs) would be such a terrible thing to vendors in our field. In the process of doing so, I guess we could also think about how we could re-structure that data to be easier to integrate with other systems in the business. Anyway, food for thoughts.

Tomorrow is the last day of the conference. I will attend another full-day class, this time on technology architecture for BI.

Today I was in new territories. I took both sessions on Data Requirements Analysis and Data Profiling for Reengineering, Knowledge Discovery, and Information Quality, both given by David Loshin. We spent the morning discussing (well, not me!) data discovery, data assessment, and data requirements. We went deep and in all details into David’s methodology. It felt like speaking a foreign language at an upper-intermediate level: I cold understand all we talked about, but missed several of the subtleties.

I was very impressed with the level of scrutiny data warehouse and BI people put in their assessment of the data quality. I think we should pay a little more attention about that in Web Analytics. Of course, we are talking here (BI) about a way more complex world. Still, I think we should do a better job at questioning our data. Looking at all that can and should be done with data assessment, I even wonder now how the ASP model is good for us, if we can’t have immediate and full access to the data/logs. I mean, when you stop thinking about it, what validates the accuracy of the results in Google Analytics? Yes, the brand. That’s it! No access to the data; got to trust Google that the numbers are good. Not acceptable to a professional web analyst I think. This means that the Omniture and WebTrends (OnDemand version) of this world should make it very easy to access the logs.

The afternoon session was basically focusing on what can go wrong with data. Wow! a lot, as you can imagine. I can’t go through the details of all that. It would be too long and I haven’t digest everything yet (if I ever will!). I was just extremely happy to be exposed to all that.

In early evening, I listen to a vendor panel featuring SAS, Business Objects, Oracle, MicroStrategy, IBI, and Cognos. Besides the very deep technical stuff their customers in the room asked them, they all agreed that text/verbatim analysis was very hot. Most of them are choosing to acquire vendors in that field instead of developing that capacity.

OK, that’s it for today. Tomorrow is going to be even deeper in the data. I wonder if I will emerge from this week in one piece…

OK, now I’m beginning to understand why people who were telling me that 6 days was exhausting were smiling. I’m starting to feel it in my bones. This was another day packed with information. The day started with Michael L. Gonzales again with a session called “HandsOn-Statistical Analysis for BI”. I have known that I needed to revisit statistics for a while (on Jim Novo’s advice), and I got confirmation again today. Actually, Michael was telling us that BI people as well tend to forget their importance. So, we went over exciting stuff such as descriptive statistics, linear regression, and control charts. I’m not being sarcastic here: it is really cool and exciting stuff. I wish I knew more, and will definitely hit the books. When I asked him about what books could be good, Michael suggested the two tomes in the For Dummies collection, even though he does not like them in general. However, these ones seem to be particularly good for a re-introduction to stats.

The afternoon was spent with Wayne Eckerson, whose book Performance Dashboards, has been an inspiration to me for some time. Although Wayne had way too much material to be covered in half a day, we went over very fundamental stuff in dashboarding: KPIs, business architecture, system architecture, design and layout, etc. I can tell you that dashboarding in BI is very heavy stuff. Projects can run in the hundred of thousands of dollars, given their complexity. This has nothing to do with what our Web Analytics vendors call dashboard. Aren’t we tired of those meaningless graphs we often scroll down over? I actually wish they were all under the tables!

OK, you can tell my the length of this post that I’m fried. Got to go to bed early tonight; tomorrow will be spent on data requirements analysis and profiling. Gulp!