Avoiding Information Overload

By Duncan Rein
September 1, 2006

The growth of the Internet has added many lanes to the information super-highway, and this has increasingly resulted in information overload. Each day our email boxes are flooded, and there are more websites, more articles, and more weblogs updated more frequently than ever before. Indeed, it can be a full time job just to sort through all the information that is available to us, let alone to do anything useful with it.

More than ever, the ability to extract actionable knowledge from a vast sea of information is at a premium. This is particularly true in the field of web analytics. There is perhaps no facet of the web that has been more over-hyped or under-utilized. How many times have we heard the mantra that the web is so powerful as a marketing and communication medium because it is so measurable? In theory, every click from every visitor is measurable, yielding a wealth of marketing intelligence and insight that can be used to optimize virtually every variable. However, the reality for most organizations is that very little of this information is actually converted into actionable knowledge. Most of it remains hidden in logfiles, never to see the light of day.

Oftentimes, when interviewing a ministry executive during a discovery meeting, I will ask questions about current web performance. It is not uncommon for this person to call down the hall and request a web report from the person in charge of the website, and for this person to walk in 30 minutes later with a print out containing hundreds of pages of numbers. The ministry executive will usually confess that he or she never looks at their web reports because they don’t know where to begin. Again, we have the problem of information overload. There is a lot of new information that is now accessible to us, but where do we begin in trying to make sense of it all?

The Importance of Web Analytics

Yet more than ever, it is important for an organization to be able to evaluate the success of its online efforts, as the web is becoming more central to its organization’s overall communication and fundraising strategy. According to the WebTrends 2006 CMO Websmart Survey , 56% of marketing executives surveyed said that the web is now the hub of their marketing efforts. It is estimated that 34% of all time spent engaging with media is now spent on the web, and this percentage is rapidly increasing. As more people spend more time on the web, organizations are seeing the need to make larger investments in time and financial resources to effectively engage with this growing audience. As investments in the web grow, so will be the importance of tracking the return on these investments.

And while web analytics has been over-hyped and under-utilized, it remains true that web analytics represents a great deal of potential and promise for an organization to better understand its audience and to optimize its approach for connecting with it. The challenge is to unlock this potential, so that it can work for your organization.

In this article, the first of a two-part series, I will explore eight principles for instituting an effective framework for web analytics within your organization. The second article, scheduled to run next month, will outline principles for incorporating web data with other important offline metrics to evaluate the overall success of your online presence, including approaches and methods for estimating return on investment.

Principle 1: Clearly Define Success

Defining success as a first step for any project seems obvious and fundamental, but it is staggering to see how many web projects proceed without clear consensus within an organization as to how success will be measured. Is the primary goal of the website to educate people about a cause or a need, to engage people with the teaching of the ministry, to acquire names, to increase donations or to drive additional product sales? Oftentimes, it is some combination of multiple goals, and different individuals within an organization might give different priority to different goals. We’ve all seen the home pages that grow without ceasing as more messages and more ads are tacked on to the ones that are already there. This is a clear indication that the organization has not clearly defined success for their website.

To guard against this kind of digital schizophrenia, we recommend that an organization formally document its definition for online success. The document should be in bullet point form, attaching clear priority to each goal, and no longer than a page. While each stakeholder in the organization may disagree with some of the goals or their relative priority, there should be a healthy discussion around these goals. While there will rarely be unanimous agreement and consensus on every point, all stakeholders must agree at the end of the day that this is how the organization has decided to define success. This document then serves as a point of reference for all future conversations. As a living document, it should be recalibrated at least once a year, and can be adapted to incorporate new information, new circumstances, and new organizational priorities.

While it is important for a definition of success to be clearly articulated for an organization’s web presence as a whole, the same principles can be applied to discrete projects, such as the launch of a separate micro-site, or the development of an email campaign. A definition of success should be clearly articulated as a first step for any significant web project.

Principle 2: Track Metrics that Effectively Measure Performance Against Goals

This is just a fancy way of saying that the most important things to measure are the things that matter to your organization achieving its goals. Take each goal you have outlined in the first step. What are the metrics you would use to evaluate performance against that goal? In many cases, this is an elementary exercise. If a goal of the site is to encourage online donations, then the total number of online donations would be an important metric. So would the total revenue from online donations. If a goal of the site is to acquire names, then it will be important to track the number of online registrations. If a goal of the site is to educate and inform, then number of visitors, number of page views, and number of downloads of various files should be highlighted. If a goal of the site is to build community, then the total number of online members, as well as the total number of posts would be important metrics to track.

Principle 3: Make Sure the Technical Infrastructure is in Place to Accurately Track the Appropriate Metrics

This principle is also quite fundamental, but it is surprising how many new sites are launched in the absence of any time spent thinking about data collection and report generation. This is oftentimes a function of success not being clearly defined and a lack of legwork around deciding which metrics are important to track.

The lack of consistent and accurate web reporting for many organizations has also been hampered due to inadequacies in the tools available. Despite all the hype and promise, web analytics is still a surprisingly immature field. While everything can be measured in theory, there is still significant disagreement around how to define terms and how to collect data, making apples and apples comparisons particularly challenging. Even for a metric as fundamental as a unique visitor, there are many definitions (cookie-based, IP-based, etc.), and different approaches for measuring data (analyzing sessions, parsing log files).

Many affordable web reporting programs have historically lacked the ability to track key metrics, while also providing questionable accuracy around the metrics that were tracked. Advanced reporting systems, such as Web Trends, produced better performance, but were oftentimes cost-prohibitive, particularly for a smaller organization.

However, just in the last six months, we have seen a quantum leap in the availability and accessibility of advanced web reporting infrastructure, even for the smallest organizations. Not surprisingly, Google is leading the charge with its introduction of the Google Analytics service about a year ago. With Google Analytics, Google has taken an advanced reporting system (previously known as Urchin) and made it available free of charge to the general public. In order for Google to track usage on your site, some Java Script needs to be inserted into your site, but anyone with a basic knowledge of HTML should be able to set this up in 15-30 minutes. It should be noted that while the service is free, Google does glean general intelligence from all the sites that have Google Analytics installed, though it represents that it does not misuse this information. If you have privacy concerns, read Google’s Term of Service and Privacy Policy or contact Google before you proceed.

Apart from Google Analytics, other free solutions include Stat Counter and Click Tracks. And finally, there are other analytics packages, such as VisiStat, which aren’t quite free, but which are eminently affordable (starting at $15-$19 per month). Of course, an organization requiring extremely sophisticated analysis can still purchase a more expensive package such as Web Trends, but the bottom line is that budget is no longer an excuse for not having reliable web reporting. Any organization, regardless of size, can install an affordable web analytics package, and this should now be a non-negotiable requirement for every website.

Principle 4: Calculate Ratios that Provide Intelligence About Drivers of Performance

It is important to note that web analytics is more than merely generating reports, but it involves interpretation and analysis to produce actionable knowledge and intelligence. In addition to providing important information, an important goal of web analytics is to provide individuals with knowledge that will help them make more informed decisions.

An important tool for developing this kind of knowledge is to calculate ratios (also known as key performance indicators) that can provide some insight as to online behavior. For every goal defined, there will be many drivers of success. As an example, total online revenue is driven in part by all of the following: the cumulative effect of all marketing initiatives, the total number of individuals who visit the site, how likely these visitors are to click on a donation form, the percent of those people who follow through to make a donation, and the average amount of each online donation.

The calculation of ratios can be a useful tool for succinctly capturing the effect each variable is having on overall performance. To use the above example, clicks per impression for various marketing channels measures their effectiveness in bringing people to a site. Once on the site, a conversion rate can be calculated to determine the percentage of people who click through to a donation form (a conversion rate can be calculated for landing pages specifically designed to motivate a certain action in addition to the home page). The conversion rate for each donation form can be calculated (successful donations divided by donation form page views), and finally an average gift size per donation form can be calculated. Evaluating each step can help isolate the areas of greatest opportunity for improvement, providing knowledge as to where to focus investment in a world of limited resources.

For a site that is informational and educational in nature, number of unique visitors and number of page views are important primary metrics. Examples of key performance indicators for evaluating the site’s effectiveness could include number of page views per visit, number of visits per visitor per month, exit rate from the home page (percent of people who leave the home page without clicking on anything), and percent of visitors staying on the site for a minute or less.

It is clear that the number of ratios one could calculate is almost unlimited, and some of them are much more useful than others. It is important for your organization to determine which ratios are most important to calculate, based on the goals of your organization and the goals of your website.

Principle 5: Pay Attention to Trends and Changes

Particularly given uncertainties around definitions of raw numbers such as unique visitors (for instance it is widely reported that Google Analytics consistently under-reports the number of unique visitors compared to other reporting programs), trends and changes can oftentimes be more meaningful than raw numbers. Certainly, reporting that takes into account changes over time, in addition to providing a static snapshot, is much more useful.

So for instance, a monthly report might show the percent change as compared to the previous month, as well as the change with respect to the same month a year ago. The three most recent months may be aggregated into a quarterly number and compared to the previous quarter, as well as the same quarter a year ago. Finally, numbers can be compared on a year over year basis.

To visually capture trends, it can be helpful to color-code reports. For instance, one might use green to communicate increases and red to communicate decreases in key ratios. Red in bold could serve as a fire alarm communicating precipitous decreases requiring immediate attention. Again, the key point is that oftentimes trends will provide more useful information than the raw numbers themselves.

Principle 6: Assign Clear Responsibility for Report Generation and Push Information to Key Executives on a Consistent Basis

Representatives of web analytics software providers will often highlight ease of use and set the expectation that people across the organization will login on a regular basis and view reports. While this may be true in theory, and it may be true for certain individuals, it will almost never be true across an organization, even if the analytics system is easy to use. We are all governed by the tyranny of the urgent, and even if logging in takes only a few minutes, those few minutes will be squeezed on a busy day. Furthermore, even systems that are easy to use require basic training, and many executives don’t have the time or patience to learn new systems. As a result, many organizations have implemented expensive analytic packages that go unused.

Rather than hoping for the best that everyone will login to a reporting system and generate the reports that they need, we recommend an approach whereby responsibility for generating web reports on a regular basis is clearly assigned to an individual staff member or a trusted third party. Regardless of who it is that generates these reports, there should be clear accountability for making sure they are generated on a regular basis. This person or entity is tasked with pushing the right information to the right people at the right time, rather than relying on them pulling it for themselves (though this doesn’t preclude them from being trained in how to pull a report if need be).

The person or entity generating the report should give significant attention to presentation. The primary metrics should be presented first, with trending clearly displayed (e.g. color codes). Other information should be provided in logical groupings. In addition to data tables, thought can be given to displaying the information in graphic form. Finally, in addition to presenting numbers and trends, the report generator should provide some rudimentary analysis. What are the one or two things a person reading this report should notice? This should be succinctly communicated in a top-line summary.

Even if a busy executive only takes a minute to glance at a basic summary report on a weekly basis, this approach insures that important information about online performance doesn’t fall off the radar. If the web has been labeled as an area of strategic importance and priority, then consistent reporting helps to keep it top of mind at all levels of an organization.

Principle 7: Develop a Frequency of Reporting that Makes Sense for Your Organization

It is important to note that multiple reports can be developed, each with a different frequency. The frequency of how often a report is generated should be related to its purpose. For instance, a detailed analysis of which areas of the site are most popular can be very useful in deciding which areas of the site to focus investment in a world of limited resources. However, such analysis can be time consuming, and there shouldn’t be radical changes over short periods of time. As a result, it would be a waste of human resources to perform this kind of detailed analysis on a weekly basis. That being said, if an organization recalibrates and “re-aligns” its website every six months, as we recommend, then it might make sense to perform this kind of analysis twice a year. On the other extreme, a quick response email campaign will require reporting within a couple of hours, particularly if A/B testing is involved. However, it will probably not be necessary to provide this level of detail across the organization, and this will likely be a case where the person running the campaign will generate the reports he or she needs. High level information about the performance of the campaign can follow a more normal delivery schedule.

While frequency requirements will vary from organization to organization, we typically recommend the following schedule for report generation:

Weekly summary reports, showing the three to five most important metrics and ratios, with comparisons to the six-weeks previous.
Monthly reports, showing all significant metrics and ratios, with comparisons to previous month and the same month a year ago, as well as year to date and quarterly comparisons, where appropriate.
Detailed web reports, in which significant analysis is done to evaluate the effectiveness of various sections of the site and various campaigns to drive actionable recommendations. We recommend that these reports are generated no less than every six months. However, they may be generated on a quarterly basis, or even a monthly basis for extremely large organizations.

Over and above a consistent schedule for report generation, there should be a consistent schedule of meetings to insure that the data is being reviewed and discussed on a regular basis. All key stakeholders within an organization should get together on a monthly basis to discuss the monthly report, even if the meeting lasts only five minutes. Longer, more strategic meetings of at least two to four hours should be scheduled no less frequently than every six months, to evaluate the more detailed web reports and to re-evaluate and recalibrate priorities in light of the data presented.

Principle 8: Be Willing to Make the Upfront Investment Required to Develop an Infrastructure for Web Analytics within your Organization

Our experience suggests that web analytics is much more than selecting a technology package, but it also involves the development of processes and engagement across the organization. As such, it involves organizational commitment in terms of time, human resources, expertise development, and financial investment. A regular schedule for generating reports and meeting to review them goes a long way towards instituting an organizational approach to web analytics. However, smart decisions about the frequency of various reports and meetings are necessary to prevent it from becoming a drain on limited resources. Regardless of frequency, consistency is key. Reports will become increasingly useful as there is more history with which to compare current performance.

Implementing a strategy for web analytics according to the framework outlined in this article requires upfront investment. Success needs to be defined, the right metrics and key performance indicators need to be selected, the appropriate technology infrastructure needs to be installed, report templates need to be developed, responsibility for generating reports clearly assigned, and enough hours allocated to make sure that this work can be performed on a regular basis. Education needs to occur across the organization as to the importance of web analytics, and training needs to be performed for people who will be required to work closely with web data. All of this requires time.

In light of the increased importance of the web as a communication channel, every organization needs have accurate, useful, and actionable information about its online communication efforts. If your organization does not have this in place, you need to find the time and resources required to build the required infrastructure, despite the ever-present tyranny of the urgent.

Oftentimes, a third party can jumpstart an effort by helping to generate momentum within an organization and then providing expertise and experience to drive an initiative forward. Please contact us if you’d like to explore ways in which Silas Partners can help you build your online reporting infrastructure.

This article has focused primarily on approaches for capturing and displaying information captured on the web. The second article in this series, which will run next month, will outline principles for incorporating web data with other important offline metrics to evaluate the overall success of your online presence, including approaches and methods for estimating return on investment.

OTHER HELPFUL RESOURCES

Websites with helpful information related to web analytics:

If you would like to find out more about how to implement these strategies and ideas in your organization, please take a moment to let us know a little about your needs.