Constant improvement is a mantra here at ShopLocal. To that end, the team knew there was room for improvement in our popularity ranking algorithm that we call “Wisdom of the Crowds” (WoC).
The current system for determining product popularity only counted those products that a shopper specifically clicked on. There are a number of product level clicks that are counted in terms of a shopper expressing interest such as a product detail page, a buy online click or an add to a shopping list click (which are cumulatively called Expressions of Interest). Not any more. The ShopLocal team has begun collecting and counting another key piece of consumer interest data – product rollovers.
By factoring in both product rollover data AND expression of interest data, the overall Wisdom of the Crowds ranking data is all the richer and accurate. This process of collecting and using rollover data for SmartCircular sites is underway and will complete over the next month or so for all clients, SC sites and versions of SmartCircular.
At a macro level, what we have learned so far about rollover data is:
- Rollovers are on average about 5x as common as Expressions Of Interest. This means by adding them to the Wisdom of the Crowds calculation, the overall popularity ranking data should become much more accurate and more truly reflect actual consumer interest.
- Rollover usage varies significantly by retailer. No surprise given all the variation in UI/UX out there within weekly ad sites. Also some sites don’t even use rollovers. Or on other sites, the rollovers are down played. In any case, we are now tracking them all so as to extract the most useful and complete set of consumer interest data as possible.
Illustrated here is a specific product level rollover that has been triggered by a user rolling over a predefined hotspot region that is overlaid on top of the circular page view images. When talking about rollover tracking, what is actual being tracked is the number of times, for a specific product, a rollover is triggered and left open for at least 2 seconds. If it meets that criteria, it would be counted as one (1) rollover for the product.
Here is a quick snapshot of some early returns from a 6 day sample set of data across 20 retailers:
- # of total product level exposures: 1,006,078,269
- # of total product rollovers: 25,620,887
- # of total product specific indicators of interests expressed: 4,532,430
- Overall ratio of interest divided by rollovers: 18%
- Rollover trigger rate (of overall total product exposure) : 2.55%
- Interest trigger rate (of overall total product exposure): 0.45%
- Total CTR (rollovers + interest): 3.00%
UPDATE 1/21: The ShopLocal analyst team just today cranked out a nice little additional stat on this topic that is worth sharing. The correlation of interest/exposure ranks compared to the rollover/exposure ranks, on average overall is 0.6636 Meaning that rollovers are for most sites a good measure of interest. Maybe think of it this way. This correlation helps prove that the items a user rolls over are likely the very ones that they end up taking other subsequent action at a later point in their site visit.
For this first of few behind the scenes looks, we will be looking at a few of the main tools and services that the ShopLocal infrastructure team uses to ensure high availability and performance of all the various public web applications / sites that ShopLocal hosts.
The goal of this series is to give clients & partners a better, more complete understanding of all the areas that ShopLocal provides value across. In other words, for every dollar of client / partner revenue that comes in, some portion of that dollar goes to fund the world-class infrastructure that ShopLocal brings to all clients / partners. The internal team takes great pride in this set of systems, as it is not some low-budget, “server room in the office closet type start-up data center”, loosely managed, afterthought. Rather it is a best-of breed solution that is leveraging partnerships with the leaders in each respective field.
At a summary level this post will examine how ShopLocal uses:
- Keynote - An external, 3rd party availability and performance monitoring that is done from an end user perspective
- Akamai - One of the largest and most powerful content distribution networks (CDN) that ShopLocal heavily uses to cache and distribute web site content
- Rackspace - A top leader in dedicated, fully-supported web hosting / data center that also provides a powerful set of server health monitoring and remote admin controls
Keynote - Performance and Availability
- By being a 3rd party, this brings real time objectivity to any client / partner feedback and reported issues
- Configured as a proactive monitoring solution that has many levels of alerts and escalation setup to warn internal team members of potential and/or real issues
- Keynote offers the broadest and most representative set of test infrastructure which help cast the “net” as wide as possible. OS, browser type, network location, type of ISP, type of connection, geography, etc are all tested simultaneously
- Global measurement network across multiple locations and backbones gives a very broad perspective
- 2400+ measurement computers and mobile devices in 240 locations provide extensive reach and depth to measurements
- Accurate, realistic view of the end user experience. This is the most important point. Everything Keynote does is done from this user perspective. Most IT infrastructures have plenty of internal, non-user perspective monitors of server health, performance, uptime etc from WITHIN their data center / firewall. But these really only provide a very limited and often false picture of what true everyday users experience
This is a real (blinded to protect the identity of the eight ShopLocal SmartCircular retail clients) sample daily level (so hourly) report from Keynote that measures site performance across on average around ~950 data points per retailer
Akamai – EdgePlatform Content Distribution Network (CDN)
- Extensively used by ShopLocal , especially on all of the highest traffic sites and web applications
- Highly custom content synchronization process has bee built out to ensure the timely scheduling, caching, flushing and release of web content to the public
- Handles all ShopLocal hosted content images across all clients
- Is the largest CDN on the planet with well over 48,000 servers in the cloud and located within over 1,000 of the worlds networks
- Provides broad global coverage with numerous strategically located data centers in over 70 countries
- Creates the capability of being able to handle any and all incoming web traffic on an OnDemand basis. So unexpected surges or spikes in traffic are seamlessly absorbed by Akamai
- Provides real time bandwidth and request error reporting to hep monitor all inbound traffic
Currently consuming over 85,000,000 MB of bandwidth from Akamai in May 2009. The rest of 2009 is a projection that is constantly being re-forecasted as time progresses.
Over the three (3) years that ShopLocal has been using Akamai, we have become more and more reliant upon them as evidenced by the strong growth rates. The biggest part however of this growth is the really just the general public's ever growing demand for online circular information. There are just more users coming to the ShopLocal hosted weekly ad site than ever before.
Rackspace – Managed & Dedicated Web Hosting
- One of the top leaders consistently in hosting providers
- Employs an “N+1″ style of disaster recovery for key areas such as HVAC, power, network and equipment. “N+1″ means that there exists at least one additional unit that is normally not needed for regular operation but would be used if any one of the other units malfunctions
- Utilizes an extensive, multi-level security framework that includes for example a hardened and physically secure facility, dual dedicated firewall devices in front of all ShopLocal systems, no ad hoc dial-in capability into the production servers exists, proactive implementation of all recommended security patches, including anti-virus definition files, PGP (public/private key) encryption for the storage and transfer of some types of content / data, etc…
- Ensures that all production systems and related data are backed up daily
- Provides a dedicated support team to ShopLocal account
- Has developed many ShopLocal specific processes and SLAs developed over the the course of this long term partnership
- Extends 24×7x365 live support
- Constantly reviewing dashboard & threshold based monitoring of both network, hardware and system devices
- Committed to immediate response to emergency support tickets
- Offers top level certifications such as Microsoft gold certified partner
- Employees over 60+ Microsoft certified professionals many with additional Microsoft certifications
Here is a screenshot Cacti, of one of many different monitoring tools that ShopLocal and/or Rackspace uses. Others include Microsoft Operations Manager (MOM) & Site Catalyst. Cacti is a complete frontend to RRDTool (which is the OpenSource industry standard, high performance data logging and graphing system for time series data) that stores all of the necessary information to create graphs and populate them with data in a MySQL database.
Retailers, Brands and Agencies Need Proof
Measurement of ROI proves that PaperBoy (eg localized rich media display ads) and other digital formats that are circular content based are an effective medium for the primary goal of driving in-store sales. While this helps companies save money and increase top-line revenue, it also serves to:
- Develop the right budget for online advertising
- Establish the right marketing mix
- Create a performance baseline for continuous improvement of online advertising
In the past retailers and ShopLocal have turned to the following types of measurement approaches to try and close this elusive “loop” of web influenced offline sales by using:
- 3rd party research
- Web specific coupons
- Surveys
- Registries
- Store label credit cards
- Custom 3rd party studies that IRI, NPD, comScore, Nielsen or other research houses conduct
Matched Market Testing
While there is a place for the above methods, many of them are more sophisticated and costly ways of testing than matched market type tests. Rather matched markets tests are an effective method for an advertiser to understand their ROI and establish a baseline for performance. Matched market testing has been used for decades to measure ROI across many advertising channels. Methods like media mix modeling and panel studies should be considered after establishing an ROI baseline.
It's as easy as three steps to final measure the effectiveness of digital advertising on offline store sales
Making Testing Easy
ShopLocal and PointRoll have teamed up with veterans in the retail measurement space, including Nielsen and NPD, to allow for retailers, brands and agencies to measure PaperBoy and other circular driven digital mediums return on media spend (ROI for short). ShopLocal can guide the process to select test and control markets, configure campaign parameters, execute the tests and measure the results, all in a matter of a few weeks. And here is the best part. What does it cost? Free for all the consulting and study setup service. The only “price” in this equation is the cost of the online media buy, which we all hope will have a positive ROI to your business.
(4.00 out of 5)
Loading ...
Comments: None