My takeaways from La Product Conf 2019

Jeremy Greze

I participated for the first time in the biggest French conference about Product Management, La Product Conf.

There were two of us from Dataiku. Our objective was to learn about best practices and measure the areas where Dataiku can improve. Our Product team doubled in size in the previous months, there are now 6 Product Managers, 3 UX Designers and 2 Product Marketers. We are particularly attentive to how we can better organize ourselves.

We focused on more technical or B2B related talks as it makes more sense for our product.

My personal takeaways of the day:

• Discovery and persistence are probably the two preferred terms of PMs. 😊
• Our organization at Dataiku is not perfect but pretty good. (I'm thinking particularly of the roadmap!)
• The role of Product Designer is still not clear to me. Maybe because it makes less sense for us?
• I returned with a lot of ideas around PM analytics.

Here are now my detailed notes of the day, speaker by speaker.

A final word to thank the organizers for the quality of the event.

Launching Airbnb Experiences & Front

Nate Abbott (@nateabbott), Head of Product at Front, spoke about product iteration and how to build a successful second product.

A successful company is the succession of several "product next acts". A "product next act" is a new layer on your product(s). It can change the company. He gave some examples: Apple with the iPhone, Amazon with Prime and their marketplace, Adobe Photoshop with their subscription version.

Nate told us about his career path and what he has learned to build a successful product thanks to "product next acts".

• viable market : make sure you have a market-fit on a viable product. He learned it the hard way with his first startup on the tour operator market
• adjacent possible : have a product that is related to current customer needs and feasible.
• iterate with customers : be as customer centric as possible. For example, at Airbnb, when they launch a "next act" "not only booking a place but full trip experience" they started with several days experiences but customers actually wanted few hours because they already had/have plans during their travels.
• persistence : keep trying. At Front, they have a very customer centric workflow, everyone in the company receives customer feedback. It helps for iteration with customer feedback. But they released a major release and their NPS dropped. They just needed persistence because you don't get everything right away.

BTW, and it was not mentioned during the talk: Mathilde, Front CEO, has a blog with great content.

Why your users lie to you

Angie Born (@tripelle) is Head of Product at Scope and asks the question why users and customers lie during product research.

Cognitive dissonance is central. We self-justify a lot of acts. For example, if we ask how often you go to the gym, people generally over value their answer, maybe because they want to feel better, or because they actually went more than usual the previous week.

How do we work around this when doing research as a Product Manager?

Averaging: you can ask a question you already know the answer? (how often do you...) And you can find a biais.

We often ask questions around attitude and social & subjective norms. This explains behavioural intention. But that's different from actual behaviour. We should focus our studies on observation of actual behaviour. Example: it is hard to quit smoking even if you have the intention.

Allow for complexity in your surveys. Don't just ask for "did you know about ... ?" with 2 answers (yes and no), allow several answers with options ("it was mentioned by someone but no written content").

Be careful with long surveys that can be annoying.

Some people never give the best grade (for example 5 stars out of 5) even if they don't have an idea how to improve the situation.

There is no trick to detect (physically) if a user lies.

There is no difference between B2B and B2C. She mentioned a big lie in B2B she had during a user interview, a guy describing that he used a spreadsheet for making decisions but she realised after a while that there was actually no spreadsheet.

Diary studies can be an alternative to NPS studies.

Additional resources: "Theory of reasoned action" by Martin Fishbein. Dan Ariely's research on honesty, "Choice blindness" studies by Petter Johansson, Carol Tavris and Elliot Aronson on Cognitive Dissonance.

Roadmaps are dead ! Long live roadmaps !

Bruce McCarthy (@d8a_driven) is an entrepreneur, speaker, author. His talk is about product roadmapping.

There are many discontinued products (for example Google products); does it mean we are not good at predicting product success? Why do we continue to make features? We build, ship, build, ship, etc. The roadmap gets a lot of blame (many example of articles, personal situations, ...).

What a roadmap is and is not?

A roadmap should not be a project plan, a list of dates. It should be about the why first, a communication about the strategy and intentions. Roadmap manages outcomes (what's changed, customer success, objective), project or release plan manages outputs (what you produce, features). There is no specific tool for roadmaps; this can be stickers, slides, spreadsheets, Trello board...

The 5 components of a good roadmap:

• product vision : how a future world will benefit from the product when fully realised
• business objective : the goals the product will accomplish. it is about business metrics.
• timeframes : sequencing and guidance on timing, but with flexibility. It is okay to have "what we are working on now, what's next, what's for later". You may add a disclaimer that the roadmap may change any time.
• themes : problems to be solved in order to realise the product vision. You list themes and problems to solve, and you don't know how long it is going to take to tackle them. One example: Vempathy Trello board (link to add) with ideas of implementation; they don't know which one will be implemented.
• disclaimer : things might change.

How to gain alignment on your roadmap

First one-to-one alignment, then team alignment; allow for modifications so that you can say later "our plan" and not "my plan" when you present it to executives.

Q&A:

• Sometimes a date is committed to, in this case you should just ship what we have at that date (v1), and later we will improve (v1.1).
• With stakeholders, you should not list features into a theme/objective; you should talk about objectives and problems to solve.
• don't do several roadmaps, just do one high level themeable roadmap. Tell the same story to everyone; for example, add extra details in each case for a specific stakeholder.

Additional resources: Product Roadmaps Relaunched (O'Reilly), productculture.org

Product Manager or Product Designer? Perspectives on evolving roles

Jeremy Barre (@jeremy_barre) is Lead Designer at Drivy. His talk is around the role of Product Designer.

The goal is to build the best product for the users. Easy to use, well crafted, valuable.
Design thinking is a two-phase process: discovery and delivery. Too often we have a PM at the origin, we add several developers in the loop, eventually we have a data driven approach and polish the UI (at the end). We end up with features that do not solve a problem and that are too generic, etc.

A Product Designer has a holistic approach. It is not only about visual design but about product thinking. Product Designers are more focused on technique and business than a UX Designer.

Is it the same job as Product Manager? Not exactly. PMs need to sync more with other teams and stakeholders, Designers have more time to iterate on the product experience and to build the best product for users. When a startup grows, it is better to have more defined roles and associated goals. To work successfully together, they should co-lead a project with the designer being more involved during the delivery.

Designing your Product Playbook

Sebastien Phlix (@sebastienphl) is a Senior Product Manager at Typeform and presented their Product Playbook that they use to document past mistakes and best practises.

A few years ago, they released a mobile app (Typeform Lite) that failed (no real usage after installation). Lessons they learned:

• Avoid the term "MVP" because it is a misleading term. They now use: earliest testable, usable, lovable product. They run test interviews based on sketch files, then build it and monitor usage, and iterate to fix the small issues.
• Separate discovery from delivery. Take time to ask users, to work on the discovery, even if it coms from leadership. They do design sprint over a week. The more uncertainty you have, the more time you should spend on discovery.
• Be explicit about what you do. They have a list of tactics/techniques for each step of the 2 previous lessons so that they can explicitly state what they do. (see photo)

Additional resources: Sebastien's reading list

How to be a Data-informed PM

Amber Van Hecke works at Atlassian as a Product Manager and has a background in data analytics.

Looking at dashboards is a first step, but not enough because it it will lead to mistakes.

You should understand the entire data flow. Where the data comes from, what is and isn't collected, how it is transformed, and what's on the dashboards at the end? Create test accounts to see how all this works (for sure there are inaccuracies/bugs). Usually, there is no one looking at the all flow, so a PM probably should.

What metrics to have a look at?

All. You can create groups (activation, collaboration, ...) to make it more readable.
Once you have a holistic view, you should lead a data driven strategy: make metrics accessible in the organisation, use data driven approaches in feature development, etc.

Amber was given an OKR: increase second week active users (wk2au). This metric is a count of users that signal activity in the second week.
This metric is not a great indicator of a user finding value, which was the initial objective of the OKR. Also, it is a vanity metric, a single mass emailing can drive traffic but does not mean anything.
So they had a look at alternatives, in particular to measure Frequency and Longevity.

Solution 1: Quality User Scores: an aggregation of weekly activity scores (see photo below). The problem is that it takes 4 weeks to get it.

Solution 2: within the first 14 days, a user performs at least one key action on 3 separate days. Adapted from Trello (2 in 7) Metric

Thanks to these metrics, they found determining key actions leading to active users. (see photo below) Then, they promoted people to do these specific actions. They double checked the results with other teams to make sure they made sense. For example, they founded that cloning a repository was important, and it was also visible in the page views of the help section.

At the end, they were successful and had a 20% increase.

They have a monthly metrics review with all product teams.

Data Analysts are shared between Product teams, they don't have all the knowledge of product managers, so you need to work with them; that's a significant portion of PM's time.