Contact Form Case Study

Overview:

Problem: Due to high ticket volume, the operations team was unable to triage and resolve time-sensitive support tickets.

Team: CTO, Customer Success Manager, Program Manager, Front-end Engineer

Role: Product Manager supporting the Operations Team

Timeline: 1 month

Understand the Problem:

Possible Finance is a mission-driven financial technology company that provides small-dollar, credit-building loans.

As the number of new customers increased after the company expanded to several new states, the operations team began having trouble managing the ticket volume, especially resolving time-sensitive support tickets. The ticket backlog could number in the thousands of tickets per day.

Because of delays in resolving the time-sensitive tickets, some customers would experience delays with submitting payments on-time (which could negatively effect their credit score), face delays receiving their loan deposits, submit additional tickets, and subsequently leave negative reviews on social media and app stores. These factors impacted cash collections, the customer experience, and the public reputation of the company.

After investigating the issue more deeply, I also learned that many tickets were 'self-help' tickets, or tickets related to issues that the customers should have been able to resolve themselves without the intervention of an agent (changing their payment card or updating their email address, for example).

This case study describes the product management process to solve the above issue.

Solution Ideation:

I organized several discussions with the operations team to discuss the issue and ideate potential solutions. I focused solutions that would mitigate both the difficulty triaging time-sensitive tickets and the high volume of tickets where customers could resolve the issue without intervention (the 'self-help' tickets). I also researched the ticketing solutions of lending apps and other customer facing apps such as Brigit, Dave, Earnin, Lyft, Uber, and Varo. We generated the following potential solution ideas:

  1. Dedicate ops time to triaging tickets into priority groups (non-product solution)

  2. Sort incoming tickets into priority groups using filters based on keywords

  3. Build a contact form natively into the app that allows customers to select their issue before submitting the ticket

  4. Use a chat bot with a question tree to triage support tickets into priority groups

  5. Embed a contact form into the app using the CRM (Zendesk) to allow customers to select their issue before submitting the ticket; sort tickets into priority groups based on the selection

An example of a similar app's contact page

Solution Selection:

Due to recent state launches, an aggressive growth plan, and the small size of the company, minimizing engineering time was a high priority for this product. Rating the potential solutions for value vs. effort, I determined that embedding a Zendesk-based CRM contact form into the app would require the least effort for the greatest impact. This solution would also minimize engineering effort to build and to maintain, as once the form was embedded in the app, it could be modified and updated by the product team.

The additional advantage of the contact form solution was that it could display blocks of informational text before the customer submitted the ticket, potentially giving the customer the help they need and delecting the ticket.

Assumptions:

After deciding move forward on an MVP for a CRM-based contact form, I produced a list of assumptions that would be tested with this solution:

  1. Customers will be willing to take the extra time to navigate a menu to select the issue they are experiencing.

  2. Customers will select the correct issue from the menu (rather than make an arbitary selection).

  3. Customers will take the time to read a short text block describing how to resolve their issue.

Building a Hypothesis:

Using the 'we believe _____ will _____ because _____' hypothesis model, I outlined the following hypothesis that the MVP would test (blocks of informational text would be added after the initial MVP was validated):

"We believe that if customers can select their issue when submitting a ticket, the operations team will be able to resolve time-sensitive tickets within 24 hours because tickets for time-sensitive issues can be filtered separately and responded to first."

Defining Success for the MVP:

Before beginning the MVP, I worked with the stakeholders to define a minimum criteria for success (MCS). We decided that a successful MVP would need to meet the following criteria:

  1. The ops team is able to immediately identify all time-sensitive tickets after the ticket is submitted.

  2. Tickets are correctly categorized by the customer at least 90% of the time.

  3. Reduce self-help ticket volume by 10%.

MVP Execution Plan:

I divided the MVP into tasks and created an execution plan:

  1. List all potential issues a customer could face (this list was large: there were around 150 known issues a customer could experience)

  2. Group issues into 'time-sensitive' and 'non-time sensitive' categories

  3. Group issues into 'self-help' and 'non-self help' categories

  4. Organize potential issues into subject groups and create a menu hierarchy using customer-facing terms

  5. Create the menu system in the contact form using Zendesk's Contact Form builder

  6. Create an internal binary ticket tag for the ops team to mark whether or not the customer selected the correct category when they submitted the ticket

  7. Create a filter view in Zendesk that separates 'time-sensitive' and 'non-time sensitive' tickets

  8. Deploy the contact form on the website and the app; test the form to make sure tickets submit correctly

  9. Monitor ticket volumes to ensure that tickets can enter the backlog appropriately; spot check individual tickets, and search for keywords like 'can't submit' to catch potential issues with the new form

The issue list was created in an Google Sheets document based on input from members of the ops team, customer success team, technical QA team, and product team; internal ticket category data; and bug reports.

Each issue was placed in a category menu that was up to seven levels deep. The menu would give between one and five options per level, where an initial option would be something like, "Issue with my payment card", or "Issue with my credit report", followed by a more detail selection such as, "My loan is not marked as paid off on my credit report", until the exact issue could be selected. The menu was created with a focus on clarity and ease for the customer, using non-technical, customer-facing wording such as 'debit card payment' rather than 'interchange transaction'.

Evaluating the Results:

One week after the form was deployed, I reviewed the ticket data against the previously determined MCS.

The form proved to be effective in filtering time-sensitive tickets into a high-priority ticket queue. The ops team was able to respond to these tickets immediately after they entered the queue, so the first criteria was passed!

The second MCS concerned whether or not customers would select the menu option that corresponded with their issue. To gage success for this metric, I look at the percentage of tickets that the ops team tagged as 'categorized correctly' using the internal flag. According to the flag, 92% of tickets were categorized correctly, which passed the criteria required for success by 2%.

With the first two MCS validated, we were ready to iterate on the product to validate the third.

Iteration:

Since both MCS were passed, I organized adding help text to each drop down category to give customers the information they need to potentially resolve the problem themself. This required writing around 150 blocks of text, reviewing them with the operations and customer success team for technical and tonal consistency, and then adding them to the form.

Analyizing the ticket data and conducting follow up meetings with the ops team also revealed a few areas where there was room for improvement on the product:

  • From the 8% of tickets that were categorized incorrectly, I learned that there were issues that did not a have category in the contact form. These additional categories were added to the form.

  • The ops team mentioned the need to send multiple messages to the customer to verify their identity, state of residence, and preferred payment account. To save the ops team further time, I added these as required fields on the contact form.

Final contact form

Example page of the final version of the form

Results:

Overall, the product was greatly successful. The ops team was able to identify and respond to time-sensitive tickets, and the overall number of tickets per week dropped by 10% on average. But this metric did not tell the whole story: because the number of active customers grew significantly week-over-week, I used a ratio to determine how much the new form impacted the number of tickets relative to the number of active customers. With the week-over-week ticket counts as the numerator and the total number of active weekly customers as the denominator, the relative ticket volume dropped from 5% to 2.5%, effectively cutting ticket volume in half.

Case study links:

Bank Partnership Case Study

Contact Form Case Study

OfferUp Case Study