Designing a Brand Metric: Trust

Overview

• Helped a new team identify goals for a performance metric, and designed metric to quantify a successful brand and the team’s impact on brand improvement

• Developed a working brand metric based on primarily solo research in six months

• Created and executed a multi-phase research plan managing diverse stakeholder goals

Background

I was approached by the newly formed The Motley Fool Brand Team to measure the success and identify areas of improvement for The Motley Fool brand, and by extension, the Brand Team itself. The Brand Team knew they wanted their metric to reflect the role that trust plays in a customer’s relationship with the brand, but did not have a clear idea of what that would look like. I leveraged my background in psychology to help them define their goals. I knew that trust was a psychological variable, so I approached this challenge as if I was developing a measurement of the variable of an individual’s trust in another person, but adapting the materials to the unique challenges involved in trusting a financial media company. I designed a multi-phase research plan to answer these research questions:

• How have other researchers measured this variable?

• How does our measurement need to be different to make sure it works for The Motley Fool?

• Do people we know have high levels of trust in The Motley Fool have high scores on the measurement? 

For the majority of my trust research, I checked in with the Brand Team and the Trust Metric Review team to ensure that my research continued to support the goals of teams across the Motley Fool. The Trust Metric Review Team helped by providing feedback, advocating for the needs of diverse departments, and occasionally completing tasks to further research such as sitting in user interviews or executing a statistical analysis at my direction. The Trust Metric Review Team was comprised of 7 colleagues from the following teams:

• Brand (3)

• Business Intelligence

• Prospect Marketing

• SEO

Methodology

Desk Research

When exploring a trust measurement, I did not want to reinvent the wheel. I knew that a brilliant scientist or two must have work on trust in companies, so I began my research plan with desk research to leverage the expertise of others. I quickly found the Trust Project at Northwestern University’s Kellogg School of Business, a treasure trove of research in the intersections of trust, society, and business. Kent Grayson’s research demonstrated that cultivating trust is challenging, and that trust is comprised of three subdimensions: Competency, Integrity, and Benevolence.

Academic research doesn’t always cleanly apply to the real world. I also collected customer research from the last five years to understand what The Motley Fool already knows about how customers think about trust. I used this information to map out the current customer brand journey, and how the specifics of trust are distinct at each phase. It was clear that two subdimenions of trust would need to be added to a metric to fully capture a user’s trust in The Motley Fool Brand:

• Using investment advice from The Motley Fool to inform investment decisions (Product Use)

• Trust in the stock market’s continued average growth potential based on historical trends (Institutional Trust)

Charting out the customer brand journey with research from the Marketing, Investing, and UX Research departments also allowed me to see the gaps in understanding, which I continued to fill in over the next six months of research.

Pilot Interviews

The abstract nature of trust as a concept makes discussion cognitively difficult for the average person, so I wanted to test out some of my questions about trust before I asked users to focus on trust for 45 minute interviews. I piloted trust questions in two different projects: a brief exploration of trust and perceptions content writing with asynchronous prospective members through UserTesting, and digging into the context when members brought trust up in unrelated exploratory interviews.

Exploratory Interviews

The next phase of my research involved applying what I learned about asking tricky questions about trust, and understanding the the roots of trust present in our member’s stories. Allowing customers to talk extemporaneously about trusting brands allowed me to validate that the subdimensions of trust framework was applicable to our business, through the eyes of our customers. After consulting with the Trust Metric Review team, three audiences were identified as the key areas to explore brand trust: prospective members (10 prospects), mid-tier members (9 members), and upper-tier members (4 members). This enabled us to learn the common themes of trust that are present across our membership levels, and which themes of trust are distinct to different groups. Examples of questions I asked to understand customer’s relationship with trust in financial companies include:

• How do you define trust? What does it mean if a company is trustworthy?

• What does it mean for a company to care about its customers?

• Tell me about a time where you felt you could trust The Motley Fool. What happened? If possible, can you show me?

• Tell me about a time when you felt your trust in The Motley Fool was undermined. What happened? If possible, can you show me?

Common themes emerged from the interviews:

• The subdimensions of trust framework was appropriate measure of brand performance for The Motley Fool

• Most members spontaneously identified The Motley Fool as a trusted brand, before the brand was brought up

• Connection with The Motley Fool’s analysts play a large role in member’s trust in The Motley Fool

• Markers of transparency, such as showing work for investment recommendations, played a large role in prospective member’s trust in The Motley Fool

Pilot Survey

The next phase involved writing a trust metric based on what we had learned, and gathering quantitative data to understand how the findings from our qualitative research scaled to customer populations.

• FoolFest Study: Using an intensity sample of members attending FoolFest, a members-only conference of the most valuable customers, we

• Pulse Study: We worked with the Business Intelligence team to add our questions to our weekly metric tracking survey for two weeks to understand baseline user trust across our membership levels
  
  

Brand Perceptions Survey

I designed a survey to measure perceptions of our members in key areas of the business that were identified in interviews to likely be correlated with Trust in The Motley Fool’s Brand. Working with the Trust Metric Review team and the Business Intelligence team, we created a plan to recruit a random selection of our email lists of prospective members, members, and former members, incentivizing unengaged groups to respond to the survey using a raffle for $100 Amazon Gift Cards. Given the constraints of time and the attention spans of the survey population, the Trust Metric Review team helped me identify the highest priority variables to measure, which are indicated by bold type:

• Attitude: General

• Attitude: Business Analysts

• Attitude: Customer Relationship

• Attitude: Community

• Attitude: Content

After conducting the planned correlation analyses using BlueSky Statistics, I learned that trust was positively associated with all areas of the business. The red squares indicate the weakest relationships, suggesting that variability exists within users’ experiences - users might have high levels of trust generally, but are having less consistent experiences with the product suite. This shows that Product Attitude was a huge opportunity for improving user experiences that drive trust in The Motley Fool.

Outcomes

I created a metric that measures individual trust on a scale of one to five, one meaning no trust and five meaning high trust in The Motley Fool. Customers are presented five statements, and instructed to rate the extend to which they agree to each statements on a Likert-type scale. The statements each capture user sentiment on different sub dimensions of trust in The Motley Fool, identified through desk research and adapted using the findings from contextual user interviews:

• Competency

• Integrity

• Benevolence

• Product use

• Trust in the stock market

At this point, the priorities of my team were redirected by leadership, so I helped the brand team identify a business intelligence colleague who could help me finalize the statistical analysis to move this research across the finish line. I presented the results and the following high-level recommendations for strengthening The Motley Fool’s Brand:

• Focus on opportunities to for our analysts to connect with members

• Find opportunities to reach out to former members with high trust scores - they are reengagement opportunities

• Measure trust every six months to ensure that The Motley Fool brand is maintaining and improving trust in members 

I also helped the Brand Review team identify future directions for research when I was no longer able to continue working on this project. Follow up research included:

• Working with colleagues in the Machine Learning department to apply brand trust information to a customer retention model, in order to continue to predict the financial impact of improving brand performance

• Identifying colleagues in the Business Intelligence department who were able to infer relevant trust recommendations to individual departments across The Motley Fool

• Allowing the data set to be scrutinized by colleagues undergoing trainings in data literacy, enabling fresh eyes to further test trust hypotheses and understand the customer demographics present in different levels of trust