Consumer Data, (De)Coded.

Our Process

Have you ever wondered why a supermarket is arranged the way it is? There are obvious pairings; toothpaste beside toothbrushes, dairy products next to eggs, and cheese not far from the wine section. What about less obvious pairings?

Enter the famous beer and diapers example. Why are these two products often in close proximity to one another in retail stores?

The answer to this question goes back to the early days of Market Basket Analysis. This type of analysis leverages large sets of receipt data to determine which items are frequently purchased in the same visit, ultimately helping retailers to better understand relationships in purchase behaviors. Researchers noticed that beer and diapers were frequently appearing on the same receipts, and they observed both the demographics of these purchasers and time of day they were occurring, in order to better understand this non-obvious relationship. Based on their observations, they hypothesized that dads on a late-night diaper run were also stocking up on beer. Consequently, retailers that were ahead of the curve began to place diapers next to the beer aisle to provide greater convenience and suggestiveness to consumers and saw sales rise.

While emergent product relationships can inform brand and retail strategy, tracking consumer experiences beyond the point of purchase has been a blind spot for analysts. How consumers talk about their experiences and purchases, including their sentiment and motivations, involves complex and multifaceted reasonings: they are much more involved with the products they care about - they discuss why they like them, buy them, and how those products could better meet their needs. Receipts are black and white, only revealing what people buy and not the why.

To address this blind spot, Social Standards has developed an intricate system that captures and tracks topics that frequently co-occur on social media. Just like the co-occurrence of beer and diapers on receipts, the connection between two or more products or topics emerges over time. Coincidences crystalize into clear and stable connections once patterns are established by the sheer number of occurrences. To this end, Social Standards aggregates social media posts and clusters meaningful subjects into subgroups so analysts can discover and interpret salient perspectives or lifestyle choices.

Social Standards effectively treats social media posts as a kind of ‘lifestyle receipt’ that provides access to the consumer mind to access these unfiltered insights. By relying upon the collective intelligence of online communities, we begin to crowdsource market research. Generally speaking, collective intelligence is understood as being the accumulated brainpower of a group and an emergent process that depends upon interaction. Since there are countless groups online, conversations on social media are extremely diverse and their analysis requires a delicate approach that retains contextual elements. Conventional survey methods try to understand these feelings and motivations, but their approaches are limited by smaller sample sizes and likely skewed as they don’t measure people in their natural environment.

Our team uses text analysis to detect differences, and patterns of differences, en masse. With social media text, we remove contextual commonalities or norms, to reveal a collective group’s differentiating features or preferences, among other collective groups or the larger whole. With a large collective social media as a standard background, we use discriminatory analysis to enable comparison between a smaller collective group and the standard background. The collective element makes possible the comparison between aggregate conversation about one brand versus another, such as comparing two Beverage Alcohol brands’ competitive landscaping in the hard seltzer category. Underneath that aggregate conversation is the essence of collective intelligence - the sheer amount of expression of individuals that reflect the commonality or norm that is common across any day, at any given time, in a dynamic manner.

While our method removes random bias and noises to ensure the accuracy of our data, we introduce a layer of contextualization to the data. This layer of contextualization stems from our ability to segment and dissect social media data to look at it through various points of views. Imagine the whole of social data as a globe, your view of the data through the angle of Asia versus North America from within the globe would certainly look very different. Influences from factors such as climate and culture would make a particular choice of cuisine more salient in one part of the world versus another; these subtleties gradually influence people’s habits and perspectives. Moreover, views from within the globe would be drastically distinct from viewing the entire globe from outside. These differences from contrasting perspectives bring up different relational information, for example, the range of topics that frequently co-occur with “self care” in the Personal Care vertical, compared to the Food & Beverage vertical. In turn, these relational information leads to extremely contrasting conclusions. Here, these differences are desirable.

Because of the dynamic and adaptable layer of contextualization in our data, the usage of our data can be adopted to specific market needs by anyone within any level of a corporation, instead of being restricted to a particular usage. In the same management consulting company, the data that is valuable to an analyst who focuses on the Food & Beverage market would be distinctly different from another analyst in the Beauty market. The data provided to them would be the same, but the topics in relation to each other within the Food & Beverage market versus the Beauty market are impactful in their own ways. If the Director of the same consulting firm would like to examine how consumer conversations on sustainability vary across Beverage Alcohol, Personal Care, and Health & Wellness markets, our data can cater to such interest by providing context to the data through measuring the collective intelligence relative to appropriate market vertical and benchmark. This could reveal key insights, such as sustainability having an association with luxury in Beauty and local in Beverage Alcohol.

Equipped with real-time organic consumer data backed by sample size in the millions, our structured data can help you make informed decisions by discovering the unknown unknowns.

To see the power of this approach in action, please check out the two examples below:

Example 1: Strategy Executive @ Fortune 500
Example 2: Data Analyst @ Popular DTC Brand