00:35 Marcin: Hi, my name is Marcin Kokott, and welcome to Product Odyssey, the series where we explore how to build fundamentally better products. Today with me again are Mike and Maciej. Hi, guys.

00:46 Maciej: Hello.

00:47 Marcin: Today we'll be using your experience from more than 150 delivered projects to discuss one perspective, and that's the perspective of failure in projects. We tend to not focus on that because we assume that we plan for success, but failure is always a part of every project, and we need to account for that. Today, let's focus on very specific failures. These are the most common and obvious areas of failure or risks in projects that CEOs or CTOs might not often consider, but from our experience, they're significant risk areas.

01:28 Marcin: Let's find out if we have any insights or takeaways for CEOs and CTOs. So, let's start. Who wants to take the first area?

01:40 Mike: I'd say that maybe one of the failures that isn't as common but is extremely dangerous is not being aligned with developing what you need. That's actually not focusing on the impact of your project. Just building the project, providing the output, but not actually impacting the business benefit.

02:10 Maciej: I think it's difficult to do because while our actions can actually create some kind of output, our efforts are outputting some results. It's really hard to align those outputs with an outcome that we as a business want to achieve and, as a result, provide the impact that we want to make on the world.

02:34 Mike: Yes, you always need to manage this in the correct way, and it's not always possible to measure the impact directly. There are cases where you can do this, but I know a few projects that had real issues because they were focusing on generating the code and even new features. But the problem was that the vision wasn't really there. The vision was being built while the project was already being developed.

03:08 Mike: Of course, you don't want to have a waterfall approach where you plan everything and then just code it. But on the other hand, you also need planning. To me, it's surprising how often agile is treated as an excuse not to plan at all.

03:25 Marcin: Let's just stop here to help me understand if the problem area is aligned between us. So, the problem is thinking about self-optimizing a little bit, right? Thinking about how fast I'm delivering, how fast I should be developing this functionality rather than thinking about what the impact is in the end, right?

03:45 Mike: Yes, exactly. It's much easier to measure the output rather than the outcome or impact. So that's the trap that's very easy to fall into, and once you start doing this, you have additional pull because maybe you already have a team that is waiting for new features because you already engaged them, but your product manager doesn't have all the answers yet, and he needs to build something quickly. So, it's a trap that you fall into deeper and deeper, and you have less time to really think about how this impacts the project.

04:25 Maciej: Yeah. It's also really easy to fall into this trap because we have our effort under control. We can try more. We have our outputs under control because we can deliver something different. So, this is something that we have direct control over. But later down the value stream, when we talk about the outcomes of our output or the impact in the end, this is where we don't have direct control over it.

04:57 Mike: Yeah. And very often, you think that it's actually impossible to measure the impact, but in cases where it is possible, you can measure things like how much the new features that you built or the new actions that you took influenced the growth of the company. You can have a goal for everyone to see, like maybe you want to grow three times per year. It's okay to translate those goals to the team and to build the additional actions that are easier for people to control and influence out of this main goal.

05:36 Mike: The biggest mistake you can make is not to convey this. Very often, you can convey the impact you're aiming for, how it links to the outcome, how it conveys to outputs, and how you can actually try to achieve it. Sometimes there is a situation where you cannot measure the impact. Then you can measure the outcome and go a little bit lower in this hierarchy.

06:09 Maciej: Yeah, but isn't this the reason why projects can fail? Because they focus too much on the output, too much on the efforts being made, and not on the final impact that they want to achieve because it's harder, right?

06:24 Marcin: I need to stop you here because we are mentioning impact goals, and this is very generic. Just to stay on the core of the problem, can you define an example of the situation we're exactly talking about? What is the mistake in terms of functionality, approach, or team structure?

06:47 Maciej: Yeah, let's say you want to develop a new feature for our product. Our effort in this case would be to write some code, create documentation, and maybe design the product. These are our efforts. The outputs of those efforts are the code written, the new feature implemented, some integrations, design docs, and maybe some feedback gathered. These are things we have direct control over. But the next step, the outcome, is less tangible. The outcome is a change in the behavior of our clients that we want to affect with this new feature. As a result, they will come to us, pay us more for our product, and recommend our product to others. This is the impact, the revenue stream we are targeting in the end, right?

07:50 Mike: Maybe I can give an actual example. Let's take a simple project we're working on, our uniforms package, an open-source code we built some time ago. The goal was to have a lot of downloads, aiming for a few million. That was the impact we were aiming for. We conveyed this to the team working on the application. The outcome we wanted from the tasks we were performing was to make the package as friendly to developers as possible, easy to use, and for developers to have a great experience and recommend the package further. That was the outcome we aimed for, but then you translate it to output: good documentation, good code quality, and architecture. If you don't convey this altogether, you can easily get results that won't give you the final impact.

08:58 Marcin: Okay, so I'm trying to deconstruct that, and I'm seeing three areas you're mentioning. One is that if we're developing a feature, it's not about describing what the feature does, like step A, B, and C, but understanding why we need this feature, not another feature, or why we need this feature done in a specific way. So that's one. I need to have a goal for the feature and the impact, right?

09:26 Mike: I look at it even differently. If you have a great team on your project, once they know the impact you're aiming for, they can come up with ideas you didn't think about and find better ways to achieve the impact.

09:41 Marcin: Exactly, and this is the second point. You cannot do the second step without the first. If you don't know why you're developing the feature, you can't assume the team will help you define how to do it better, achieve the impact, and advise you on the best approach. The team needs to understand the business goal and vision behind it, not just respond to the next feature to be developed.

10:17 Marcin: This all boils down to the third point, having one team concept. It's not just the owner's responsibility to define features A, B, and C but to share the journey. In most cases, we do experiments. We don't know what the impact will be, so we need to focus on experimenting. Experimentation assumes everyone participates and provides feedback. These three points address this common failure and risk area.

10:51 Mike: Yes, exactly. It's complex because you can look at your project and business through different lenses, like the business model canvas. You find things during development. You don't always have the right impact or it changes with time. It's about managing how you communicate this. The biggest mistake and failure, based on our experience, is not trying to do this at all.

11:32 Mike: Yeah, you are destined to fail or have very little chance of success.

11:38 Marcin: Right. Don't ask why we do it, just do it efficiently. Just deliver functionality. That's a common mistake. Maybe it's obvious to me, but not to the development team, and they're missing critical feedback and good ideas. Any last takeaways? If this is the main risk area, what should I immediately do in my current projects?

12:17 Mike: That depends on the project, but you need to know what you're aiming for, your goals, and look at your project through some lens, like the business model canvas. Plan based on this and convey it well to the team. Get feedback and adjust your plan. Measure it and adjust your actions based on the impact or outcome whenever possible.

13:15 Maciej: One more insight could be that when you're planning your work, don't focus only on how to do something, but why you want to do it. Keep this why in the back of your mind, in your backlog, in your user stories. Most often, you may not need to do it to achieve the reason behind it, and this is an advantage.

13:41 Marcin: Yeah, from my side, it's very close to what is already known in the market. Customer development approach or lean startup is common, and we need to learn from that because it's not always implemented. If you develop the functionality, involve the team in understanding that delivering functionality is not the end. Measure what happens after that. Does it work? Measure and make corrections, then plan the new functionality. It's hypothesis-driven development. Everything we do is not defined and finite. It's an experiment, so learn, fail, and the team needs to be prepared to sometimes scratch functionality they've delivered. It's closing the loop, hypothesis-driven development, and team involvement in this experiment and scratching if needed. Let's move to the next point. Who wants to go next?

14:56 Maciej: I think the next typical failure in products could be not putting enough attention to DevOps. Yes. And by this, I don't mean the classical DevOps like CICD pipeline or automated tests, although those are very important. I think if you miss the actual reason why you use DevOps and how it is the lifeblood of your product, depending on what your product is.

15:27 Maciej: For example, for software as a service platforms, the processes involved in the broader picture of what we call DevOps are crucial. Missing important pieces there could actually cost you your product in rare cases. By DevOps, I mean something broader, even business continuity management.

16:01 Maciej: DevOps doesn't have to be only about shipping faster or automating tasks that people otherwise do manually. Good DevOps practices enable you to move faster and achieve product-market fit before your competition. Iterate quickly.

16:33 Maciej: For more mature companies, already finding their product market fit, they need to ensure they can continue delivering. In case of downtime or failure, you need processes that allow you to recover quickly. You need to know how to behave when these situations happen because they will happen.

17:19 Marcin: I see. Reformulating the problem, we see situations where clients in the startup phase think they'll deal with processes like crisis management or business continuity later. They think, "If we have success, we'll think about it." But then it hits you, and it's too late. Even if you're in production, with a few users, you're not thinking about it. But it's always too late when it hits you. Do you want to extend on that problem a little bit?

18:20 Mike: Yes, it's common on almost any project we work on. DevOps is often omitted at some level. You think about DevOps when you're launching live or scaling for a big event, but then you also prioritize other things first. It's hard to convince the founder or investor to invest in DevOps planning on a regular basis.

19:00 Mike: But you get the biggest return on investment by planning ahead. You don't need to do everything at once. You don't need a full DevOps team optimizing things from the start. Initially, you need a plan to act when you grow faster than expected or encounter problems you can predict.

19:27 Mike: At a minimal level, you need to know whom to call to solve your problem. With a plan, you can handle crises better. You know what to tell your users and investors when things go wrong. Without this, it's chaos. Often, you prioritize other things and then hit chaos when something happens.

20:15 Maciej: For many companies, these processes are "nice to have." Nobody opposes them, but this mistake happens repeatedly. Not enough attention is given to it, and this can come back to haunt you. Even if you're moving fast and breaking things, good processes let you move faster and break fewer things. There's always something that could be done a little better. We have leverage here that many companies aren't using enough.

21:14 Marcin: Yeah, from experience, there's an approach where people think it's a one-time activity. If we're selling DevOps, they think, "I need to invest." But that's the wrong way of thinking. It's not adding a cost but saving costs in the future because if not addressed, it becomes a problem. The misconception is that it's a one-time activity. I need to plan it, and it's huge, so I'll plan it for later. But the product backlog example shows how to deal with lots of things to implement and prioritize. It's probably the same here, treating it as an element in the product backlog.

22:19 Maciej: Yes, it's more a process thing than a one-time thing. DevOps isn't just setting up CICD or continuous deployment. Adopting a DevOps or DevSecOps mindset changes how you behave and helps you be aware of issues that might otherwise slip through.

22:56 Marcin: Right, it's an incremental approach. It also touches on infrastructure costs. It's not a one-time activity. It's not about sitting down and deciding on AWS or the server setup. It's about having time to focus on decisions, defining when to make decisions, and deciding if something is a one-way door (irreversible) or a two-way door (reversible).

23:29 Marcin: If it's a reversible decision, try it out and move back if it doesn't work. If it's irreversible, plan and focus on it. Timing and awareness are critical. Don't delay it, and know you can do it incrementally.

23:57 Mike: Yes, exactly. It's about at least having time to plan, and that's easy and cheap. When things happen, you also have some time to react if you've set up your plan. If you know how to react, you can handle things more cheaply.

24:15 Maciej: This is also related to risk management. Many companies take this into account only from the risk management perspective. It's fine, but many companies manage risk this way.

24:29 Mike: It's about the return on investment and preparing a plan that will give you the return at the stage you're at.

24:37 Marcin: We've landed on some insights. One thing from experience is that incremental approaches are simple. At the beginning of the project, it can be simple stuff, not even hiring one FTE but giving someone a task to assist and think about the area. If it takes more time, build on it. You don't need to hire more FTE or build a team immediately. Have a plan and think about it.

25:10 Marcin: You can also get expertise from outside your team to verify your plan and scale your approach. You don't need a DevOps-focused company or your own team right away, but it's good to know which tool to use and how quickly to do this.

25:48 Maciej: It's always beneficial to have an outside perspective on your processes, to recommend things you might have missed. Even if you have a complete platform team, it's beneficial to have an open culture where people can contribute to the broad spectrum of those.

26:15 Marcin: To summarize, what should I do if I want to focus more on DevOps infrastructure and business continuity management processes, depending on the stage of my development?

26:35 Maciej: Sit down and consider whether you're putting enough attention into the broad DevOps idea in your product. Are many developers or others mentioning areas that need improvement and you're ignoring them? Are people happy with their developer experience? Are processes going smoothly? Are you able to deploy on time?

27:07 Maciej: If so, congratulations. You're in the right place. If you sleep well after deploying on a Friday evening, everything is fine. But if not, rethink how you can improve this and put it into the core philosophy of your teams' work.

27:39 Maciej: Maybe you can be a little bit better, achieve a little bit more next time.

27:46 Marcin: Good hand.

27:48 Mike: First, check if you're already at a place where adding more business continuity management procedures or DevOps could help you. If you spend a lot of time and money on on-call support and servicing your project, it's probably too late. Include some automation already. Use your backlog. Plan ahead. Delegate someone to have ownership of DevOps and monitor it regularly. Plan it in your schedule, backlog, and approach to the project.

28:40 Marcin: You've actually stolen my idea as a takeaway because that's a mix of that. Be aware of these processes and know you'll need to implement them sooner or later. Plan for success, and if you want to plan it, it needs to be part of the backlog. Treat it as part of the backlog among the functionalities and epics you want to implement. Treat it as an investment with a reward connected to it. There's a cost and a return on investment. It's easy to do and treats it against the risks, so you know what and when to do it or what risk you take if you don't do it.

29:17 Marcin: That's my takeaway. We've talked about two main issues. We have time for at least one more common issue we've seen. Anyone want to take that?

29:34 Mike: I think that would be the management of technological debt.

29:39 Marcin: Oh yes.

29:39 Mike: Yes, it's a broad topic, but not managing this consciously is a common issue. There are a lot of articles saying that project complexity kills almost half of the projects. I'm not sure if it's true, but it's definitely something that requires significant investment in analyzing how you restructure the project. If managed correctly, you can often avoid it.

30:16 Marcin: Okay, but we need to define this as a problem. Technical debt is well-known. Complexity grows, and technical debt accumulates. What's an example of the problem connected to that?

30:32 Maciej: Well, you'll see when you have problems with technical debt. People will tell you about it all the time.

30:39 Marcin: If I ask developers, they will always say we need to deal with technical debt.

30:41 Maciej: Yes, but it's true. There's always too much technical debt, and developers always want to try new things and rewrite everything. That's one factor, but it's not the most crucial one. Metrics like DORA can help you see how much you are delivering. Value stream management methodologies allow you to measure actual impact. This helps you see when things start to fall apart, and one reason could be technical debt.

31:14 Mike: It's good to explain a few common examples. One case is starting with a prototype meant for a month or three months, with the goal to rewrite it later, but there's never time to rewrite it. So, you extend it, and it becomes your production application without planning to adjust it. Your time to deliver new features gets longer and less efficient.

31:58 Mike: Another case is when technology changes and your tech gets outdated. Sometimes it's okay; applications can last 10-20 years without changing the tech. But sometimes something new appears that speeds up development significantly. In extreme cases, delaying the rewrite of part of the application can end up being just two months of work, speeding up development three times.

33:06 Mike: You need to make these decisions through correct analysis. You should always look at it as checking the return on investment on certain actions. Not preparing a plan for how to manage technical debt, how to monitor it, and auditing your application from time to time is the main mistake. You don't prioritize it.

33:42 Marcin: That's interesting. Technical debt is well-known, but you're putting a different lens on it. It's not just being aware and dealing with it but dealing with different strategies. For example, planning a rewrite in three months is okay, but ignoring it for sure is a bad practice.

34:14 Maciej: Technical debt can be a good thing if managed correctly. Let's assume we have a product before product market fit, and we do everything top-notch quality, not moving fast and breaking things, everything with 100% test coverage. We're not moving as fast as we could. Technical debt is like loans; we take them to achieve a result faster.

35:02 Marcin: I compare it to training, creating fatigue, and destroying your muscles to get better. You need to manage the fatigue level. Technical debt will always be there, but you need to measure and deal with it effectively. It's not getting it perfect but understanding it will need to be dealt with later with a rewrite or extensive time.

35:48 Maciej: Yes, like any debt, you have some interest on it, and you need to manage it.

35:53 Marcin: Exactly. Monitoring technical debt isn't that hard. You can often audit your project within 10-20 hours, have an architect look at it, and monitor how things look. The recommendations should be based on the return on investment. For example, investing 20-30 hours for refactoring to get back 20 hours per month.

37:05 Marcin: We have a great example with a current customer where we have technical Fridays, a time dedicated to improving code, processes, and the project. Allocating time and agreeing that this is part of the process allows us to measure the impact.

37:38 Mike: In this project, it makes sense. But on many projects, it's premature to have one day for this. You can increase or decrease this time based on need.

37:53 Marcin: This is a slider again. Use it as much as you need, like fatigue in training. If it's critical, and you're moving fast with a huge technical debt, experiment with the amount of time needed.

38:15 Maciej: You can put different efforts into the topic. But the deadly mistake is not thinking about it at all. If you skip it, it's not on your radar, and you're not planning.

38:31 Marcin: Yes, you don't see the spider; it's hidden and more dangerous.

38:35 Mike: When handling technical debt, there's also an anti-pattern. Sometimes you dedicate a team to build part of your application completely differently to improve the architecture. Often, you get other people than the original developers, and this approach misses the knowledge of those who built it initially. You spend too much time building a new architecture, not involving the original team. This can lead to problems, as you don't use the internal knowledge.

39:06 Mike: Involving people who built the application, getting expertise from outside if needed, but using internal knowledge is crucial. This can give you the result of handling this well.

40:16 Marcin: That's a good perspective. Any comments?

40:19 Mike: I completely agree.

40:21 Marcin: I would also cover the basics: measuring defects, speed of development, and doing good code reviews. This gives good indicators of technical debt levels. It's not one metric, but it's a safety net to know if this is a problem or not.

42:34 Marcin: Exactly. You can't measure technical debt directly. It's not about bad code but a process thing. Information silos, lack of documentation, all contribute to technical debt. You need to look at the process, interactions, code reviews, and other factors to mitigate it.

42:56 Marcin: It's like the story of cutting trees without time to sharpen the ax. It's always about how much you want to maintain and do maintenance to move faster later. Invest now versus how much time you'll spend running.

43:39 Mike: To summarize, have the correct structure for the phase you're in. At first, you might not need to focus on it much, but build structures, tools, and roles to handle it over time. Audit projects regularly, plan optimization and handling technical debt based on return on investment, and involve the team in planning restructuration.

44:22 Maciej: Technical debt can be good or not bad if managed correctly. It depends on the project phase. Technical debt will always creep in, but complexity needs to go somewhere. Manage it with procedures.

44:37 Marcin: The complexity is inevitable. Measure it and accept it, but don't accept any level you have. Prioritize speed at times but plan for rewrites if needed. Address it regularly if you're mature enough to avoid it growing and causing issues.

44:57 Marcin: Yes, pay your debts always. That closes the third point. I guess that's all for this episode. It was very insightful, and we'll probably continue discussing more risks and failures in future episodes. Thank you for watching. If you have any comments or feedback, feel free to reach out. Don't forget to subscribe to learn about new episodes.

47:41 Marcin: Thank you very much for watching. See you in the next episode.