Coach Response vs User Retention Experiment

Naluri.life

Naluri is digital health app that focus on delivering holistic health program for physical and mental health.

Objective

Experiment and analyse correlation between coach response time to users activities to increase engagement and retention

Project Overview

Coach response vs user retention experiment is a series of experiments that focusing on giving users feedback or acknowledgement as soon as users did activities.

We set the target response time to be 1 hour after users did activities and we compare acknowledgement coming from coaches vs customer service (CS).

Contributions

Design the experiment together with operation team
Analyse the experiment result with data team

Results

- Responding faster doesn’t translate to higher retention rate, responding once a day has a higher retention rate than responding every hour.
- There's no significant difference between CS and coaches in acknowledging to the activities
-In general, we see drop on weekends (with or without acknowledgement) and it gets back up on Monday. (We also don’t give acknowledgement on weekends.)

Impact

From this experiment we continue to give daily acknowledgment to users and this eventually became a feature in the Coach Platform, named coach recommendation.

We also change the SOP for coaches to attend users even on weekends. Also, they have coach assistant to help them reply with a generic message.