Case Study: How would you investigate a drop in user engagement?

Problem statement: You show up to work Tuesday morning, September 2, 2019. The head of the
Product team walks over to your desk and asks you what you think about the latest activity on the
user engagement dashboards. You fire them up, and something immediately jumps out:

The data shows the number of engaged users each week. Yammerer defines engagement as having
made some type of server call by interacting with the product. Any point in this chart can be
interpreted as “the number of users who logged at least one engagement event during the week
starting on that date.”

You are responsible for determining what caused the dip and, if appropriate, recommending
solutions for the problem?

Answer the following questions:

 Do the answers to any of your original hypotheses lead you to further questions?
 If so, what are they and how will you test them?
 If they are questions that you can’t answer using data alone, how would you go about
answering them (hypothetically, assuming you actually worked at this company)?
 What seems like the most likely cause of the engagement dip?
 What, if anything, should the company do in response.

Solving the case:

To give the solution to the problem let us consider some points due to which the engagement of the
user could be drop:

Probabilities:

a) Long Holiday: because of holiday which lead to drop in traffic

b) Technical Issues: Website down, Update issue, Server glitch etc.
c) New product in market
d) Some particular page is not working
e) Internet down
f) Sites cookies are giving error
g) Campaigns or events reduction

Solution to the Hypothesis/Probabilities:

a) Long Holiday:

Verification and Solution:

 Compare drop in user engagement vis a vis state holiday in the country
 Check if the user engagement is down in that specific period only.

b) Technical Issues:

Verification and Solution:

 Compare with the historical data with any bugs or technical fault in the app (if mobile app
available) or website which led to decrease in user engagement.
  Check whether the update is provided by company and user is not aware about the update
of the app
 SMS can be send the user to do the necessary updations.

c) New product in market:

Verification and Solution:

 Check whether any new competitor in the market.

 If found so, then find the services provided by the competitor.
 Compare it with our features with our app/websites.
 Develop the more refine services to the users.
 Float new features so gather more audience.

d) Some particular page is not working:

Verification and Solution:

 Check if any particular page on which user visit mostly not working properly.
 Check past few weeks and analysis if there was some problem in it.
 If found so then do the necessary action and inform the users about it’s smooth working.

e) Internet down:

Verification and Solution:

 It may be possible that in some areas the internet is down.

 Due to which users are not able to logged in.
 Check the data of the states if any internet connectivity issue was there.

f) Sites cookies or notification are giving error:

Verification and Solution:

 It is possible some time that websites cookies are blocking some pages.
 Check whether is server was refreshed in past days.
 Check if users are getting to much cookies.
 Or users are getting to much notifications.
 Try to refresh the server