People Driven Development

Episode 17: Outnumbered and Outgunned
===

Eddie Flaisler: ​[00:00:00] Hey everyone, and welcome back to the PDD podcast. This week's question gave us an opportunity to make this a very special episode. We needed some expert help and we're extremely lucky to be joined by Nuro's Robin Atwood, a Silicon Valley veteran, and one of the most capable engineering leaders you'll meet in your career.

Robin Atwood: It is great to be here. Thank you so much, Eddie, for the opportunity and Morgan, it's great to meet you.

Eddie Flaisler: The pleasure is all ours. Robin Atwood holds, we would say one of the most interesting roles in the industry at the moment. For the past six years, she's been at Nuro, a startup focused on autonomous zero occupant delivery vehicles. They recently transitioned into a licensing model for their autonomous driving technology. N uro is probably the epitome of more with less. Building and deploying custom autonomous vehicles is capital intensive, especially when navigating the complex regulatory landscape. Capital which, unlike the well established parent companies of its competitors, Nuro doesn't have in such [00:01:00] abundance. And somewhere at the heart of this world stands a woman responsible for two groups at the core of Nuro's technical operations: enterprise support engineering, and enterprise architecture and automation. Groups, which simply put have to keep this resource constrained and thereby even more mission critical compute alive and moving along the insane volumes of data required to operate AV fleets. Before Nuro, Robin was a lead software engineer at NASA AIMS research center specializing in drone simulation, and she has 17 plus years of software engineering experience before that.

Robin Atwood: Thank you very much. That sounds pretty amazing. I think I might hire myself. I'm pretty stoked.

Morgan VanDerLeest: That sounds like an excellent plan. I am for one, really excited for this one. So with Robin's extremely relevant background in mind, let's jump into our question.

Dear PDD, for the past two years, I've been head of engineering at a small FinTech startup founded in 2021 during the Covid crisis.

Unfortunately, we haven't done well. [00:02:00] What started as a promising idea with strong early investor interest has yielded little result with almost no cash left and no prospects for private funding. Our founders, and honestly I, still believe in the company's potential. And recently we came across an interesting opportunity: a small public company whose business is nearly gone, but who isn't ready to give up their public listing, approached us.

They're offering a reverse merger. We would merge into them, our management team would take over and we would receive most of the shares. So everyone kind of wins. We'd gain immediate access to capital markets, shortcut the IPO process, and hopefully regain some credibility while their shareholders get to monetize what's left with their value and avoid de-listing, which nobody wants on their record.

It all felt promising until the technology due diligence came. We failed miserably. The public company wanted to make sure we had solid technical safeguards and operational practices to ensure data security, uptime, safe code deployments, and good control over how our infrastructure is set up and changed.

We do not, we barely have monitoring. I have one engineer who does some infra and [00:03:00] security work part-time and a tired, demotivated product engineering team who's already overwhelmed just trying to keep our existing revenue going. For this to happen, we need to be ready in a few months at most. What do I do?

Eddie Flaisler: Well, what I find very interesting about this question is that it is truly one of those cases where the generic leadership advice on ruthless prioritization is, at least on the surface, not an option. If the business is going to survive, they need to both keep revenue flowing and completely reshape their tech ops posture at the same time. This makes it one heck of a challenge. I will say that I don't want this anecdotal instance to give air cover to leaders who simply refuse to prioritize even when there is room to do so. Squeezing the team to the last drop instead of making hard choices isn't bold leadership, it's just sloppy

Morgan VanDerLeest: No disagreement here.

Robin Atwood: Absolutely

Eddie Flaisler: Queue the intro. Let's do this.

Morgan VanDerLeest: I am Morgan.

Eddie Flaisler: And I am Eddie.

Morgan VanDerLeest: Eddie was my boss.

Eddie Flaisler: Yes.

Morgan VanDerLeest: And this is PDD: People Driven [00:04:00] Development.

Eddie Flaisler: So the way we always start our episodes is by looking at the question a little more closely. To that end, I think it's worth starting with some clarity on what our listeners trying to achieve. Technology due diligence is a critical part of any merger acquisition process where technology is core to the business value. It's somewhat similar to what many of you may know from vendor reviews to ensure alignment with your needs as a customer, but on a much larger and deeper scale. Instead of just validating a specific system or service, in due diligence, you're evaluating the entire company's technology environment. So how secure it is, how reliable it is, how scalable, and how well it can support the business going forward. When it comes to security, due diligence looks at things like encryption practices, access control, incident response readiness, vulnerability management and governance structures. It's not just about checking for technical gaps, but also assessing whether the company has the organizational muscle to maintain a strong security posture as it grows. When it comes to [00:05:00] operational resilience, which Robin will speak to, due diligence focuses on areas like infrastructure reliability, system scalability, disaster recovery, change management processes, and how well internal operations can support growth, audits, and future regulatory requirements. Especially in resource constrained environments, the way technical operations are structured can make or break confidence in an acquisition. With that context in mind, we'd love to hear from you, Robin. What should an organization be prepared to show during the technology due diligence process like this?

Robin Atwood: Great question. I l iked how you broke it down. In my mind, essentially it is vital to take a long view, to take ownership of the current situation, understand the current situation, and prepare to own the outcomes over time. Figure out where you need to be and build a roadmap to get there.

Leverage extended context and insights by conferring with your team members along the way. Articulate the vision of where you wanna be, but above all, you need to be relentless in your pursuit [00:06:00] of this long-term goal. And make progress by meeting the challenges that your team faces today. Operational teams often struggle to see beyond the tactical situation. Issue Y happens, we check our playbook, we try things, eventually address the issue. Knowing your goal is fault tolerant and reliable systems, as a manager, you should not be jumping in and fixing the issues. Your job is to understand why these issues are occurring, how frequently, and how to fix the causes of those issues. A problem that never arises is always better than when we have to work to resolve. The benefit here is that by doing this, you gain the trust of your team members and you contribute to making their jobs less frustrating in real time. Leveraged well, that trust will empower them to do the same thing. When you do this properly, you show your team the value of audits, of inspections, of metrics and monitoring.

You prove it to them step by step. They see the value of this and become enthusiastic collaborators. They adopt the values you have embodied [00:07:00] and displayed as their own. This results in lasting cultural change, the most enduring impact you could ever hope for. In my experience, regulatory compliance, scalability, reliability, and meeting regulatory requirements fall into place by using this model. Read the regulatory requirements in advance, study them. They're the map to where you want to be. Bottom line, industry regulations exist so that regulators can be assured your company is operating well and is prepared to meet future challenges. Small interactive changes by themselves won't get you there. But instilling cultural values through earned trust and collaboration has a cumulative effect that is enduring and incredibly rewarding. Rather than trying to figure out how to handle the challenge yourself, you have a team of well aligned people whose solutions to their portions automatically align with the overall goals: a self-organizing system. Your listener, really doesn't have the luxury of time. In that case, I suggest they [00:08:00] jumpstart the process, sharing requirements the team must meet and brainstorm with their team at the beginning. Find ways to achieve the most important results quickly, celebrates the wins along the way.

Work to structure meeting the requirements in ways that also improve the general team experience, and show them how they help. Take time every week or two to demonstrate and celebrate progress with your team and brainstorm with them how to meet the next challenge. In this manner you'll find the output of your team far exceeds the sum of individual contributions.

Morgan VanDerLeest: Awesome. Thank you for that. You know, I'm not sure the next question will help our listener very much, but I have to ask. Hearing you speak, what strikes me is that many of the requirements you're describing don't feel like things you should only do when chasing a certification. You see what I mean?

Don't I want an audit trail for my financial transactions from day one? Don't I want to avoid having my company name become synonymous with mishandling personal data? I understand that some requirements like data sovereignty or obsessive tracking of every [00:09:00] code change can feel a little over the top when you're just getting started, but others definitely do not.

What has been your experience working on the side that's often labeled non-core and has to fight for priority even though it is critical to the business? Humanize it for us. When leadership doesn't actively prioritize your domain, where are they typically coming from?

Robin Atwood: Happy to try to field this one. And, just a quick aside, I'll be answering not specifically from Nuro, but just from my own broad experience, just about everywhere I've worked. When you're in a startup mode, it's easy to focus exclusively on agility and product development. Apparently tangential concerns like reliability and security can often be deprioritized. The cost of retrofitting these into an existing product, however, can be exorbitant. I strongly believe that this is where most startups on the verge of success fail. The required investment at a late stage is simply insurmountable. This is why one of my first priorities when I led a team of engineers at Nuro was to champion a [00:10:00] nascent infrastructure as code initiative and continue to give it priority throughout Nuros cloud transition. Having a repository containing all the details of every cloud deployment helps us redeploy and scale more rapidly and efficiently, allows security audits, et cetera. Essentially, it's vital to focus on the end goal. In this case, leveraging the capital raising capabilities of a publicly funded company. You're seeing a crisis in terms of needing to rapidly evolve and change your tech stack to meet its opportunity. And one thing I've learned in my career is to never waste a good crisis. You're going to need to rapidly redeploy your tech stack in a new environment. You could either brute force migrate a system at a time, or you can take the time to rebuild things better. It is imperative that your leadership be educated about the risks and rewards. As an aside from personal experience, now is the time to get them on board with infrastructure as code. Ensure they hire the right [00:11:00] consultants or full-time employees with the correct superiority experience and infrastructure as code experience that you need. And buckle down and get to work. Where compromise is required, Lean toward audibility and security. Bottom line, Morgan is absolutely correct. These are the things you need to succeed as a company and compliance should come as a product of the investment that you are making on a daily basis.

Eddie Flaisler: I really like your point, Robin, and also I like the idea that you brought a little earlier, which is you start from what regulatory compliance will probably require. You start from what the future might hold and it sounds unnecessary. It sounds like we don't need it right now, but you're not necessarily building those things that are not needed right now. You're building your stuff in a way that will allow you to expand. This proves itself time and again as necessary. Anyhow, to take this off track a little bit because I also want to touch on security. One thing I've always found challenging as the [00:12:00] leader of a product engineering domain is distinguishing between point asks - so things like upgrading a vulnerable dependency or fixing a piece of code that the security analysis flagged - and issues raised last minute that are core to the design or even the product requirements.

You see what I mean? It's very important to address core nonfunctional requirements of the system. But also there's a question of timing. I've worked for a company where during a final security assessment before a big release, there were privacy concerns about our strategy for sharing location data and questions about whether the key exchange scheme we chose to secure the connection between front end and back end was appropriate given the nature of potential attacks on such a feature. Now, here's the thing. These are super thoughtful questions, but addressing them can take weeks if not longer, because they pertain to how the feature was fundamentally designed. They cannot come up as a last minute ticket two days before the release because the impact of the business can be catastrophic.

Morgan VanDerLeest: So what did you do in this case?

Eddie Flaisler: [00:13:00] Well, what any mature management professional will do, Morgan. I threw a tantrum. But then once I calmed down, we did the only thing you can do if you don't want to cancel the release: compensating controls. One, we reworked the rollout plan to limit who's included in the feature flag and gets to use it first. Which by the way is why I always force my frontend devs to componentize their code so it's easier to add feature flags at any point, even if right now it's not needed. Two, we audit trailed, and logged the hell out of that feature and defined some conditions we would moderate to detect potential abuse. Three, we implemented some restrictions on user session length and permissions to minimize the blast radius if something does happen. And lastly, when it was once again agreed to all stakeholders that we want to move forward with this feature long term, we prioritized doing the proper fixes sooner rather than later.

Morgan VanDerLeest: That's very scary though, isn't it? I noticed that when people need to sign off on a difficult security [00:14:00] decision, they find creative ways to avoid that because of the major responsibility that comes with the agreement itself. I.

Eddie Flaisler: I am very curious to hear what Robin is thinking here. And I'm far from being an expert, but I do know that from a compliance perspective, not less important than the actual measures is documenting the decisions properly. I honestly believe it's never too early for that. Even when you're a three people team working on a POC, if it's going to be used by others. We can make mistakes and hard prioritization decisions, But it's important to show that we took into account known risks and what we put as a stop gap to minimize those. There are some excellent open source threat modeling frameworks out there that anyone can use for free to guide and document their decision making. you build the muscle to formally document your decisions as an organization, it feels much less scary to make mistakes as an individual because the process truly protects you. Hashtag psychological safety. What are your thoughts, Robin?

Robin Atwood: Yeah, those online and OSS resources are incredibly valuable. I'd add [00:15:00] that you need competent, vetted security professionals, not part-timers, with an up-to-date context on threat mitigation strategies. And never be afraid to invest in tooling to support them. They can help surface and document risks.

You may never have been aware of threats that are outside of your experience or context. Tooling can help quantify the risks, demonstrate in a cool objective numbers, the velocity and trajectory of threats that the company faces, and provide you with the ability to protect your company from a dynamic and rapidly evolving environment. Regulations and standard compliance are by design behind the times. It takes time to make reasonable decisions based on things that have happened and their impact. The reporting The reporting for these should come as a natural product on your due diligence in protecting your company. Bottom line, if you're doing things right to begin with, compliance is largely effortless.

If you're not, figure out the underlying reasons for compliance requirements and use them to draw a map to get you and your team ahead of them. Your team will enjoy the psychological safety [00:16:00] that results and they will thank you for it.

Morgan VanDerLeest: That makes a lot of sense, Robin. To get us back to the question, in retrospect, my point about why not do it right from the beginning seems to not have been completely off topic. It adds another potential layer of complexity to our listener's situation. They didn't explicitly call it out, but sometimes the problem isn't just about what you didn't build, it's how you built what you did.

Eddie and I were just talking the other day about a situation at one of his previous employers where he was tasked with implementing GDPR's right to access- the regulatory requirement that you must be able to provide a customer upon request a full copy of all the data you have on them.

Only then did they realize that while the company's data architecture was definitely performant and reliable, the data was completely scattered and not organized by user. It just goes to show how a seemingly harmless design choice can cost you dearly if you ever need to adapt to new requirements. So to that end Robin, I think we can offer our listeners some value right off the bat. If you have an example from [00:17:00] any point in your work history of a situation where you needed to build something, but the system definitely wasn't set up to make the incremental change obvious. How did you approach thinking about the technical solution?

Robin Atwood: I've got a long answer for you. At Nuro, we were fortunate that PII has enjoyed conscious considerations since the beginning. However, However, I can share related rapid change in usage patterns and how we reacted to it. It's no secret that autonomous driving generates a huge amount of data. LIDAR point clouds, 360 degree radar, and camera coverage, et cetera, be all become enormous streams of data. Nuro uses these to improve our autonomy and particularly the safety of Nuro driver. Before a state Change lands on our fleet, it is subjected to playbacks and simulations of thousands of potential safety issues to ensure our driving meets our rigorous safety standards. In the early days when I joined, Nuro had a very small fleet. Whenever an event would occur that caused the autonomous driver to tap [00:18:00] out and ask for human assistance, a ticket was generated in Nuro's ticket management system. That occurred whether it was an analysis of a log from a real world mission, or when we ran one in simulation. It was a reasonable model. Each ticket would be interacted with by a person who would then categorize and assign work to address the problem. You see where this is going? When Nuro moved to the cloud, the number of tickets generated in the system quickly began to exceed our people's ability to keep up. Our vendor supplied ticketing system had to be scaled continuously until it became a monstrous cluster in the cloud. Unfortunately, that ticketing system was human-centric. It wasn't optimized for massive parallel ticket access and disposition. In the interest of moving forward quickly, the data from these tickets would now be imported in huge batches into proper data access systems and things kept rolling along. The flaw in this design, however, was that we were generating tens of millions of tickets and retaining them in a system designed for [00:19:00] a very different use case. Our ticket management system had become an extremely inefficient cache and workflows from multiple teams required access to that cache. I I saw the problem evolve, but focusing development effort on it at the expense of decreasing the velocity of our autonomous driving efforts was a difficult sale. In the end, we achieved the replacement of the ticket management system as cache by highlighting the velocity impact it had on that goal. By demonstrating that the cost efficiency of replacing this ticket management system as cache was so great that the work required to do so was a sound investment. Essentially, revenue generating, or potentially so, projects will always rightfully be the focus of businesses. The trick is to look for harmonious ways to marry business imperatives with opportunities. Find a way to advance progress towards new markets and opportunities while meeting or exceeding regulatory requirements. In the case of removing all PII, do it the [00:20:00] hard way for our customer first. Document the cost in time and capital. Use that as a basis to demonstrate the business risk. If N percent of our users ask to be removed, it'll cost us Y dollars. And remember, that won't make business sense to leadership right now.

Maybe they decide that it's better to take on that risk, that is their responsibility and prerogative. But then prepare to add your restructuring need to a revenue generating project later, perhaps as an add-on to a project that anonymizes PII, so it can be communicated with partners. When you pitch it as part of a larger revenue generating effort, you'll more likely get support. Be consistent and persistent in articulating the end goal. That's how progress is made in less glamorous projects.

Morgan VanDerLeest: That was a phenomenal example. Thank you. You know, something I loved was that how at each stage, your system was set up in a way that worked really well, at the time, for what you needed. There is no silver bullet that's gonna set you up for every scenario from the beginning, but having [00:21:00] that compliance in mind and a good structure in place just enables you to evolve as you need to over time.

And I think that's so, so important. How about you, Eddie? Any relevant insights?

Eddie Flaisler: Yeah. You know, I think I always found the scariest part of situations like this when you need to build something completely new to address the same problem you're solving with your current system to be, not the building, but the maintaining. You You know, I can always find someone, either internally or externally to build something from scratch at a reasonable cost, especially today. But the question is, what happens after? You immediately inherit the two systems problem, or as Robin described, with all the iterations, the end systems problem, where both the old and new systems are in use indefinitely. So logic or data is duplicated or inconsistently maintained. The team has to debug issues across both systems and testing, deployment, observability, all these become fragmented. That's why I normally care less about the engineering of the new thing we're building, and more about making sure [00:22:00] that routing and data access are centralized between the old and the new thing. I I never agree to build that part separately from what we have, No matter what. If you insist on all new, then migrating the old system to this new pipeline is part of your V zero. This is important because once everything flows through one place, what you essentially did was put a facade or a proxy in front of your new system.

Think of something like an API gateway or reverse proxy, and now you know what and who goes to which system specifically. So it gets much less confusing to handle issues, and of course over time you can migrate more and more to the new system

Robin Atwood: Absolutely. I agree. Iteration is a vital part of the business process. It is the only way to meet changing demands and changing opportunities while preserving the existing systems that got you to the place you are to begin with. And finding ways to do this - yes, and - we will continue to support this older infrastructure, [00:23:00] these older requests, while we rebuild this new one, is a perfect example of how to achieve this successfully without derailing your business.

Eddie Flaisler: 100%.

Morgan VanDerLeest: You know, one thing that really resonated with me from the listener's question was the part about, I have one engineer who does some infra and security part-time. That hits too close to home, because that strict yes or no line, you are either a security engineer or you're not. You're either in IT or SRE or you're not, is something I've personally encountered a lot.

We've seen the massive shift left on quality over the past decade. Engineering teams now own the quality assurance of their products, but somehow that shift hasn't really happened, at least in my experience, in technical operations. What I have seen instead is a not so healthy transition from SecOps and SRE owning their areas end to end, to more of a throw the ticket over the fence approach.

If you think about it, an entire ecosystem has grown around this over the fence concept. You know, tools like Dependabot or Snyk, [00:24:00] which automatically open PRs when security advisories match your code base. They're excellent tools. I use them myself, but the underlying idea is we'll just tell you what to do and you merge the PR. In my mind, that model practically guarantees frustrated, uninformed product engineering groups who potentially see teams like Robin's as these people who randomly drop tasks on them.

So because we need to be solution oriented here. My question to you, Robin, is have you ever been in a situation similar to our listener in the sense that you had no choice to engage a team outside your purview that also didn't share your expertise in order to achieve a larger goal? And if so, what were the key pieces that made that work?

Robin Atwood: Excellent question. This pops up all the time in my domain, and I've tried a lot of approaches. In my view, it starts with finding common ground. It's easy at Nuro. Everyone is here to make history, to save lives and lifetimes for everyone. Your listeners' company's goals may be different, but in the end, each company is telling a story: invest in us and

we'll make Y better. [00:25:00] I strongly recommend meeting with stakeholder teams regularly. Let them share their concerns. Encourage them to engage in active partnership. Explain that, yes, you are asking them to do things that cause extra work. Explain why it is vital. Show how these security improvements help ensure their mutual success. I like stories. They help me be mindful of the challenges a company faces while being aware of the impact of policies and directions on individuals. In this case, I invite you to imagine the team you're collaborating with as a team slogging through a deep jungle, clearing a path that will eventually become a road. The foliage is sparser on sandy areas, so they will naturally take the easier path across, along sand whenever possible. Their moment by moment experience is the progress they make in clearing a path. You encounter this team, you know they need to do more work making the path over solid soil rather than shifting sand. You can either threaten and [00:26:00] cajole them to take the harder path and then constantly monitor them to ensure they do so, only to have them go back to the quicker and easier way when you're not watching. Or you you can take the time to explain why it's vital they work harder and more slowly to create a path suitable for the road that will be laid behind them. By providing vital transparency behind the purpose of the extra work you're providing and inviting their feedback, a more collegial relationship emerges. You are no longer that person who makes unreasonable demands, but an ally helping ensure mutual lasting success.

Morgan VanDerLeest: I really loved your point about context. I feel like that's one thing that, no matter where you're coming from, even within product engineering teams, but especially across the entire engineering org, share the context.

Why are you doing this thing? 'cause if I know that, I'm happy to work with you, I just wanna know the why. Gimme the understanding here so that I can help do this in the best way with you and understand it. I feel like that's missing a lot and that ends up [00:27:00] causing so much friction just not having that shared context. So I appreciate you calling that out.

Robin Atwood: Absolutely Morgan. I've seen that time and time again. Everybody wants to do the right thing. Everybody wants to be the employee who gets it and helps the company succeed. And if you don't give them that context, they don't have those opportunities. It's such a vital thing to do and it's so easy. It makes things so much better. It's just a muscle you have to learn how to exercise over time.

Morgan VanDerLeest: 100%.

Eddie Flaisler: I could not agree more. You know, I heard a talk a few months ago by Peter Chodakowski. He talks a lot about platform teams. The title of the talk was "There's no such thing as internal customers." And the crux of it was that even though it seems like a good idea to think of your downstream teams as customers, because then you have more empathy for them, An underlying assumption we make when we call someone a customer is that they get to choose whether or not to use what we built. And And with our downstream teams, that's just not true. They will have to deal with whatever we give them. Morgan and I have mentioned here [00:28:00] before, the sigmoid model for diffusion of innovation. Basically a sigmoid curve that describes how new ideas and technologies spread. Slow at first, then accelerating rapidly, and eventually leveling off. But the type of things you build, Robin, follow a different curve. One with two peaks. The first peak happened immediately because adoption is mandatory and the second one comes later when voluntary adoption catches up. Because what you build either becomes convenient 'cause everyone else is using it or actually proves itself valuable. Why is this relevant? It's especially relevant to our listener because their missing functionality isn't this standalone thing that you built and just sits there. It has to be fully integrated into everything for you to actually pass the type of diligence our listeners describing. Your data, your infrastructure, your shared components. So So my next question is, what have you seen work and not work when trying to get other teams to actually use what you built?

Robin Atwood: Wow. Another good one. [00:29:00] Essentially, I've always seen the best results when I've worked with other teams in a spirit of companionship and shared goals. Taking Taking time to acknowledge their challenges first and really understand how their team operates achieves two important goals. First, it establishes that you're curious and interested in their team, which is vital to establishing the second: trust. When you build a trust relationship with your customer, or as I refer to them internally, you're collaborator. You open the field to thoughtful collaboration and reassessment of requirements. For example, say, another team's leader came to me and said, Hey, I need you to give me a 30 gigabits per second fiber connection. I could respond by asking them to create an IT ticket, taking the request at face value, and go about providing it for them. If however, I lean in and learn context about the team, I may discover they needed additional bandwidth because they're running a job locally, but the but the [00:30:00] consumer of the data they create is in the cloud. digging in and exploring the workflow as a collaborator, I may very well discover that providing them that extra bandwidth wouldn't be necessary at all if we took the time to analyze and understand this situation. It might be more cost effective to host the data generating job in the cloud itself rather than incurring additional bandwidth fees. Of course, maybe they really just needed the 30 gigabit line, but I will have gained the context required to be a better collaborator in the future. Better still, maybe I find out they'll need another 50 gigabit in the next three months, and I can get ahead of the demand curve at little additional cost. The same goes for software, for APIs or for selling vacuum cleaners. The relationship is the key. Maintaining that relationship, checking in with them, offering training, all of that is vital. Find Find out what they really want and show them how your solution provides it. Along the way, you'll find ways to enhance your products and meet [00:31:00] their needs and expand your market.

Morgan VanDerLeest: You know, the parallel that this is hitting for me is this feels almost like sales, right? Good sales where you're not just selling the thing, but you're coming in just to understand what's going on with a customer? Where are they going, where are they now? What do they think they need? How did they get to that understanding? And by really digging in and asking those questions and getting that understanding, going back to context, you're just giving yourself more context to be able to provide the best solution for them.

But I can tell from Eddie's face that he has a horror story that he would really like to share as well.

Eddie Flaisler: I really do

Morgan VanDerLeest: Go for it.

Eddie Flaisler: Well in most of our episodes. I like playing the innocent victim who did everything right and was still forced to adopt something he didn't want to. So I figured I should also share my own villain story. And this one isn't actually about the customers of what I built, but about the vendor I was relying on. A few years ago, I was the lead on a product that had to integrate with the API of a specific vendor that our company [00:32:00] operations were deeply ingrained with. vendor's product was good, but it was never meant to be API first. So they had very few endpoints I could actually use. The situation I was in was basically that I needed per user information and they had just one endpoint that gave information about all the users in our company. Sensitive Sensitive information, the kind you wouldn't want exposed to basically anyone. Now I thought that adding a user specific endpoint would be really easy because it was ultimately a JSON that you could extract parts of. So when they said they didn't have the resources to add the end point or even just some optional parameter based filter on the same endpoint, I got frustrated and escalated with my leadership team. Needless to say, this got me nowhere besides wasting time and getting on everyone's nerves because my employer could not afford to fight with them. So I ended up doing what I could have done all along, which was to make the call to their API from within a secure and privacy safe [00:33:00] sandbox, have the logic inside it to do the filtering and in-memory caching without persisting anything, and just query that. That vendor went on to be acquired for $1 billion and I went on to be given feedback when I applied for a promotion. So there you have it.

Robin Atwood: Hoist on your own pitard. I can't say how many times that's happened to me. Whenever I'm acting without empathy, I find that there are horrible consequences. And it's a wonderful self-reinforcing lesson about about how to manage and how to work in a corporate environment. I find that it's absolutely vital to lead with empathy and understanding, and that transparency helps others do the same.

Eddie Flaisler: 100%.

Morgan VanDerLeest: Absolutely. You know, Eddie, it's starting to make much more sense to me why one thing you've always coached us on at work was the importance of getting buy-in when promoting an initiative. I distinctly remember you even giving feedback and knocking off points, so to speak, when someone charged ahead in a vacuum without aligning others.

So I'm [00:34:00] curious from the perspective of a product engineering leader who's suddenly told their team needs to work on something, something that, as we've said, just needs to get done. How does that feel? What matters to you and how that's handled?

Eddie Flaisler: Yeah, so I don't think there's any doubt that in a situation like this, working on persuading people to use something is futile. The outcome of such a conversation doesn't matter. And in fact, if it concludes with a person being even more convinced and adamant that it shouldn't be done, you going with it still is, at least in my mind, a lot more damaging to the relationship and to people's motivation in general.

Think about it. You ask for their opinion and then you ignored it as opposed to just empathetically explaining to begin with that we have to go through with this. That said, just because I'm not trying to win your excitement doesn't mean I should build the thing in a way that makes you hate your job even more. If I don't set up a support system for when things go wrong, if I don't prioritize understanding your requirements, how you do your work and what your system needs, or if I just roll out the thing without [00:35:00] any real coordination that's on me. Behaving like that and then hiding behind, but it had to be done is just a cop out.

Robin Atwood: I couldn't agree more. And as far as far as evolving an evil empire of Reliability and respectful, collaboration, I certainly would love to have your allyship, my goals align.

Eddie Flaisler: Definitely.

Morgan VanDerLeest: I love that. So the title of this episode is Outnumbered and Outgunned, and it comes from the idea that in the functions Robin has led, there's often a mismatch between the scope of responsibility and the size of the organization. At the start of this episode, Eddie mentioned that this is one of those situations where there's not much room to prioritize or say no. I don't disagree, but I do think it'll be interesting to go a level deeper into this seemingly non-negotiable scope.

I'm curious, Robin, if you have an example where you found opportunities to simplify your prioritize even when the requirements seem strict.

Robin Atwood: Wow, lots. one of the biggest drivers for my participation in startups is that there's always opportunity to reassess and [00:36:00] improve situations. Challenging assumptions often yield surprising insights and taking the time to do so has saved me and my teams from a lot of toil. In a startup, I enjoy having the chance to ensure that processes and policies are set up to ensure long-term success and engineered from the beginning to change over time. Again. I often find the motivation for request is as vital as what is being requested. For example, I hate procedure for procedure's sake and unless that procedure has demonstrable value for my team and or stakeholders, it should be changed or removed. If it is actively harming productivity, it will be dealt with with extreme prejudice. Doing things in a certain manner, because that's how we've always done it, is nearly always a big flag for a process that must be changed or removed. For example, often support is a vital duty for any team with a user facing component or interface. And that support when provided by an engineer [00:37:00] comes at a high cost. Not just the time spent dealing with the issue, but with the break from other work the engineer was previously engaged in and presumably will be engaged with when they finished the reported issue. Sure. It's good to have the engineers involved to see what issues their end users experience, but at the same time, every interruption comes with the 20 to 30 minute decrease in productivity on a good day. And with multiple occurrences, it can disrupt timelines, delay product rollouts, and generally make life miserable. If you take the time to listen, you may find your team members don't mind troubleshooting. They may even enjoy the challenge, but the unpredictability becomes a problem. So you set up an on-call system so that one or more team members expect to be interrupted over a period of time, but they also have a period of time when they can do their focused work. You've still met the requirement. Your team is providing support, but you found a more efficient way to distribute the load. [00:38:00] Likewise, security compliance, audits, et cetera, can take up a lot of time. Or if you build the reporting requirements into your daily operations, they can take surprisingly little and maybe now your daily operations are better documented. Find and follow these procedures that suit your team's needs and change processes that don't or throw them out altogether. Understanding the why of policy and procedures helps you meet those requirements with less toil and strain. Remember, instilling these values in your team ensures continual improvement on this front.

Eddie Flaisler: Makes so much sense. I definitely think that the idea of simplifying and prioritizing inside a scope that is seemingly unidentifiable and non-negotiable is truly where engineering leadership can have a ton of impact. One of my favorite professional texts of all times is Gene Ross's "Enterprise Architecture Strategy". The idea of that book is that you don't just build stuff, you build the foundations for how decisions will be made and value will be created. [00:39:00] It really has shaped a lot of my own design thinking over the years, and it's applicable here too. There are basically three questions you need to answer. And if you take a deep breath, remain patient and disciplined for a few days before jumping into coding and get them answered properly, it truly does save you months, if not longer.

Morgan VanDerLeest: And what are those three questions?

Eddie Flaisler: Well, one, you identify the operating model of the organization you're in. It is It is so critical to understand the common theme to how processes in your company run, because this tells you what approach would be most effective given how things work today. As opposed to just saying something generic like, let's build a platform, which may not even be the right approach. There are four typical operating models. Coordination, so different units work independently, but they have to share data and synchronize activities to serve shared customers or goals. Think of a company like Microsoft. Different divisions like Azure, office and Xbox, they all operate somewhat independently, but have to share identity management. They have to share billing and customer [00:40:00] data.

The second model is unification. So units operate as one with standardized processes and integrated systems across the entire organization.

Morgan VanDerLeest: Did someone say Apple?

Eddie Flaisler: That's right. One ecosystem, one customer experience.

The next model is diversification. So each unit operates independently with its own processes, systems, and customers. So you need very little coordination. You typically see this pattern at opposite ends of a company's journey in small startups and in large conglomerates like Alphabet. And the last one is replication, meaning units operate independently, but use the same processes and systems. Think of Google Cloud service teams or AWS. Each GCP service, like storage or compute runs largely independently, but they all follow the same internal operational standards.

Morgan VanDerLeest: So while we don't know much about our listener setup, I think it's probably reasonable to say that it might be operating in diversification mode. Most small startups do without realizing [00:41:00] it. Each team has its own practices and tools because they're moving fast and solving different problems.

But as they grow, especially if they try to tighten their security and IT infrastructure, they have to shift to unification or replication.

Eddie Flaisler: And I think, Morgan, that you've just made a very critical point to this conversation. You know, we always talk here about the importance of alignment, alignment between the leadership team and the business stakeholders. And this is a perfect example. are that the type of architecture, which is probably the easiest, fastest, and regardless, even the right one, long-term to implement, will be one of unification or replication. But if your company's operations aren't interested in collaborating on changing how they work, you know, so it matches what you're building, This whole episode is irrelevant. Passing diligence is a company problem, not an engineering or a technical operations problem.

Robin Atwood: Absolutely Eddie. Establishing your company's goals, communicating those, and being transparent about how you intend to fulfill those [00:42:00] goals. How you intend to help your company progress towards those goals is absolutely vital. Without having that alignment, you're right, this entire episode is kind of irrelevant. Our caller is not gonna have a chance. But if you can help foster that level of engagement and that shared eventual goal across your organization, you have an excellent chance of success.

Eddie Flaisler: 100% alignment is everything. Alright, so with the current and target operating models figured out, your next step is to identify the organization's maturity level. Sounds Sounds ridiculous. What do you mean maturity level? We're a startup. It's actually more nuanced than that. So what maturity level actually means is how ready the organization is to absorb control frameworks without breaking. So essentially what foundational work needs to be done before you can even consider adding the capabilities that will be sought during technology due diligence. There are predefined categories here as [00:43:00] well. The earliest one is business silo stage, where each unit builds its own separate thing, like we mentioned. So you have to start by standardizing foundational tech, like identity management and logging before you actually get to implementing what the due diligence was looking for. After that, there is the standardized technology stage when there are already common tech standards, like a shared cloud platform. Now you can start enforcing things like encryption everywhere and access controls. The third and fourth ones are this mythical beast i've never personally witnessed. It's a case where all business processes are beautifully shared and everything is componentized. So different units can innovate while plugging into shared services. When a company is in one of these stages, you can directly dive into the due diligence related work. So Robin, you have worked in some very established environments. Is there such utopia? And if so, does it come with some kind of cost?

Robin Atwood: If there is such a utopia, I haven't found it. For me, frankly, it'd be [00:44:00] pretty boring. I thrive on challenges and surprises in finding out solutions to novel problems. I think the closest I've experienced to that would be NASA. They have built an incredible organization with a lot of incredible collaboration amongst different departments. It is well componentized, but it does come with some costs . I think it's pretty common for companies to oscillate between middle stages, especially as new products are introduced and new technologies and processes come into play. In my view, a stage four organization is typically too rigid to adapt, to change, and becomes vulnerable to competition. I can say however, that aspiring to that is an excellent North star. It encourages openness, transparency, sharing best practices, all wonderful things that are core to the way I work and want to work. I suspect that AI has excellent potential here as an accelerating agent. Being able to have a well trained LLM explain a code base and how to interact with it and cite examples [00:45:00] of usages is huge and that can adapt as the code changes through time. In effect, I can interact with new to me services and technology much faster and use what I create to spur greater innovation. I have experienced work environments at stage four, and to me it is far from a utopia. Innovation becomes difficult because there is a lot of operational load in making changes. Reports are made, trainings are conducted. Requests to have access to do a pull request on code outside of your scope are subjected to scrutiny. In In short, there's a lot of process getting in the way of rapid progress. In some industries that's a good thing. Aerospace, for example, and others, it's less so. Here too. I believe AI will be a helpful ally.

Morgan VanDerLeest: I love some of the call outs here for different ways that AI can be useful. What I hear outside of the technology space, it's oh, AI is just gonna start coding for us, as opposed to no, there's actually some really unique things that AI [00:46:00] can do that are incredibly useful. In this case for a typical engineer's daily life cycle or getting into new areas of the code or whatnot. I love that you called those out.

Eddie Flaisler: One thing I've found almost paradoxical since the beginning of the AI revolution in the past, I don't know, 12 to 18 months, was was that people assumed they now can be as organizations even more messy than they were because AI would cover those gaps. Now I really don't need to bother with tests. AI will figure out everything. In fact, it's the other way around because documentation and quality, and especially with models which are not always easily explainable. So checking for quality is not that obvious. And security. All these things that are more important than ever. And what I would love to see, and what I hope this industry would go towards is using AI as a multiplier to build stronger organizations. Not to do the opposite, not to neglect how an organization [00:47:00] needs to run, just because we have AI.

Morgan VanDerLeest: I think that makes total sense. To get us back on track. What's the third question?

Eddie Flaisler: So the third and final question to ask sounds somewhat philosophical, but it is in fact the most concrete than all. What actually needs to pass due diligence? This is no different than saying because we need to reinvent ourselves to penetrate a new market or make ourselves more attractive to a VC that wants to see something different, doesn't mean we have to start from scratch. You'll probably end up standardizing, I don't know, authentication data, storage, encryption, and audit logs. But do you need much else? That entirely depends on what tech you have, which touches sensitive or regulated data. Not to mention that when you're using a cloud provider like GCP or AWS, a lot of the requirements are already satisfied by the provider. So I guess my main point is let's not make it bigger than it already is.

Robin Atwood: I absolutely agree. I feel like, again, we hit that essential question of software development or engineering, which [00:48:00] is, let's define the requirements and let's understand those requirements. A problem well-defined is more than half solved, but that's where we need to, again, leverage humans with context.

Yes, we can augment them with those AI tools to help them do their jobs better. The AI should be seen as a plus to your workforce. But having the judgment as to when this is applicable, when a question should be relegated to the legal team, when you should actually involve leadership in a review of what you're finding, that is a judgment call that only a human can make, or at least that only a human can make well.

Eddie Flaisler: 100%. And you know, Robin, I love your, "A problem well-defined is already half solved". That is so true. It always reminds me how at school, whether you know, from elementary school all the way till college and graduate school, when you need to solve a problem in math, be it a proof or [00:49:00] an equation. You start from something. And 90% of the work is usually just to transform it using some knowledge you have into a different, something that actually means just the same thing, right? And then you get to a point ultimately where you realize you have very little to do, to get to the answer. So the jump is not the entire equation or the entire identity. It's the last mile. It's the final step after you've done a lot of transformation to what you already have. And I feel that is so true to every type of problem solving.

Robin Atwood: I absolutely agree. I'd say that 90% of my job comes in the IT field in knowing when to involve leadership and knowing when to involve legal. IT is where this beautiful virtual world that we build impacts the messy world with real regulations and laws with teeth in them. This is where, you know, these beautiful ideas can be dashed on the rocks of real life.

Eddie Flaisler: [00:50:00] Mm-hmm.

Robin Atwood: And for me, having the opportunity to use my experience across all of these fields. It's humbling. I get to bring all of myself to work every day to help solve this. And yes, AI is a part of this. Yes, leveraging my team is a part of this.

And I hope that the person who submitted this question is listening because they can be a part of this too. And there is nothing more rewarding than being a part of a huge integrated solution of self-organizing people towards a greater whole. There's nothing that is better than riding that wave of success until you make it.

Eddie Flaisler: I love that.

Morgan VanDerLeest: That was beautiful. All right, gang, we've already covered a lot today about strategies and tactics. For one, IT, infrastructure, and security suddenly take center stage after not being the main focus for a long time. The final piece we wanna touch on before we spice things up a bit is organizational design.

Our listener situation is that the existing team is already spread thin, just keeping the [00:51:00] current revenue going. So while we can try to optimize what we have, it really feels like one of those cases where external help would be critical. The challenge is that this isn't just a random feature that needs to be built.

What's on the table, and frankly, what much of your own work Robin has been about is core to business continuity and defensibility. How do you bring in outside help for something that critical? Have you had situations where you had to onboard external teams or contractors?

We're curious about how you structured them, how you managed their work, and any specific practices you found important in making that successful.

Robin Atwood: Well, I don't know that we have, room in this podcast for an entire 500 page treatise. So I'm going to try to respond briefly with the most important points. I've had the opportunity in my career to bring an outside contract, help and even collaborate with teams across the planet. At the base, the key to successful collaboration is clearly defined requirements.

It is vital to remember that you and your team have a huge body of context about your products and services, and even customs that your consultants [00:52:00] won't share. Assumptions must be avoided, and the terms of engagement, especially the means to establish requirements are met, are vital. You're counting on this third party to bring their talent and expertise to bear, but in the end, your company is paying the check. When you are leading such an engagement, it is your responsibility to ensure you are getting your money's worth. Before I've even set requirements, I'd like to start with a quick one page or less document that defines the business problem we're trying to solve. Keep it very simple and make sure all parties agree to that simple definition before beginning to craft potential solutions. In previous companies, I've seen that document save millions and the lack of it cost millions. Once you pass into their area of expertise with that third party, keep those communications open. If they're a good consulting partner, they will have questions. And if they don't, you should be very, very concerned. When they deliver, test and verify their solutions [00:53:00] quickly and provide feedback. If they're offering a service, make certain that you have a service level agreement that meets your needs and provides remedies if they fall short. At the end of the day, no no matter how incredibly well funded your company is, or how talented your team, will find areas where you need to rely on outside help. Learning how to manage this is key to your company's success and your own. Own the relationship with this external solution provider, the same as you own the management of your team members.

Eddie Flaisler: That's absolutely true. You know, in my mind, in situations like this, the question can either be, do do I delegate this new work to outside help, or do I delegate existing work to outside help? Whereas the core team takes on the new challenge, I actually don't think there's a clear cut answer because it depends on multiple factors. Is the developer pipeline stable and self-explanatory enough so it's easy to onboard someone from the outside to either of the work streams? Are the existing [00:54:00] code base and product spec accessible enough for someone from the outside to take over them while our team does new stuff? Do we even have the expertise in our existing team to onboard onto a new journey? And not less important, are we targeting regulatory compliance or even just a specific set of customers who might mandate that production data is not accessed by developers unless they meet certain requirements like residency in the US or even lack of criminal record? I found this latter one to repeatedly be an issue.

Morgan VanDerLeest: I can see that. Is there a typical way to address the production data constraint?

Eddie Flaisler: Yes, excellent mocking. Some retool style tooling layer that exposes obfuscated information for when you need to debug something in production, but you don't actually wanna see everything. And an effective protocol for how to let people who don't meet the access criteria still do on-call with a clear path forward in the event that production needs to be accessed directly. I know it sounds like I'm, again, describing this [00:55:00] perfect world that should have been thought through in the first place, and it's not relevant to our listener who needs to solve now, but it's just not true, especially especially with today's technology. You can spin up these things in a few very short weeks while the team is working on identifying the external contractors or whoever you'll bring on board. It's not too late to get your system to a point that you can bring someone from outside to help.

Robin Atwood: Absolutely. It's never too late, especially with the modern tools that we have at our disposal. It also helps to clarify the scope. Is this a tool that we're going to be using all the time or is this a tool to aid our transition? Maybe you don't need to spend as much work making code maintainable if it's going to be thrown away. Making those priority choices is something that is very much in your realm as a manager and something you should be paying a lot of attention to. The costs can be enormous, but the rewards are equally enormous. Miss,

Eddie Flaisler: That's That's a great call out.

Morgan VanDerLeest: So Robin, our final two questions for the day get a little bit dicey. The first one is about panic. When [00:56:00] talking about the three questions that characterize the due diligence work, we alluded to the idea that like with so many things in software engineering, you need a bit of patience at the beginning. You've repeatedly overseen domains where everything is urgent. Everything needs to happen right now, and it's probably already too late, be it incident response and emergency migration or anything like that. We're curious about your approach to stakeholder or expectation management. I think we can all agree that most people present, company included, are susceptible to stress.

And when a stressful situation hits, it's difficult to stay composed and not get in the way of those actually trying to solve the problem. What's your recipe for giving your team the time they need while not making your stakeholders feel helpless and distressed?

Robin Atwood: Well, at the risk of repeating myself, transparency. In such situations, transparency is key. For example, often you will find yourselves in situations where as a company needs you to dedicate resources to explore an opportunity while your team is already fully committed to other initiatives. First, find that common ground, that [00:57:00] shared purpose. If nothing else, those who are making the urgent demands and you should have the greater good of the company in mind. Perhaps your team has multiple stakeholders expecting deliverables and you've already agreed to a delivery date when this new task is presented. At times like these, it is vital to be open and communicate clearly. It It may even be advisable to pull the, drop everything and do this now requirement giver into a meeting and with you and your other stakeholders. Share the potentials, the risks, the consequences of reprioritization. More often than not, especially if you've done this before, and earn the trust of those stakeholders, you will find that the group can find a way to reprioritize. Failing that, of course, you can ask for clarity from leadership. I have no problem asking a VP, OC member or the CEO themselves, what in their view, should take priority. Their perspective is different from mine. Their gaze is wider, and ultimately any business decision is their [00:58:00] call. Stakeholders will see the effort. They'll appreciate it. Everyone has been in that position, but not everyone has had the benefit of shared perspective. And of course, it is important to set expectations and share shifting timelines. Again, this all goes towards building that trust, building that cohesiveness, and that ability to move in common toward a shared goal.

Morgan VanDerLeest: Absolutely. I think that perspective is exactly why we wanted to have you with us today. 'cause that so well aligns with everything that we've talked about in the past on aligning with your leadership, being transparent, sharing that context, building that trust. Love it.

Eddie Flaisler: It's just like you mentioned, Robin, relationships is everything. And that's why I always used to tell my team and also to myself, as we progress in our career, less about what we know technically and more about how we deal with people. Because at some point, especially today with AI, but even before that, you will you will know [00:59:00] less than the new intern who knows all the new technologies, who knows all the algorithms by heart from college, you will be slower. But what they won't have is your maturity level and understanding what drives people, what to do and what to not do. 100%

Robin Atwood: Absolutely, you know, Morgan mentioned this sounds a lot like sales And and to me. That's most of my job. I spend my time selling my team, selling our capabilities, selling concepts and best practices to various stakeholders throughout the company. And that is impossible without that trust that transparency gives us. That is why this value driven approach to management that I think consistently resonates throughout your podcast is so vital.

Eddie Flaisler: Thank you so much for that. To close this episode, we want to take a full step back from our listeners' question and touch a little bit more on the topic of AI, which we've been mentioning. You've worked in domains that sit either at the very top or the very bottom of the [01:00:00] stack, meaning everything actually flows through the systems you develop and maintain. Whenever new technology like AI becomes available, you're often among the first people who have to figure out what to do with it. Now since the widespread use of AI is relatively new, rather than looking for concrete examples, we'd love to hear your general advice. How do you personally think about striking the balance between supporting innovation and rapid development without becoming a bottleneck, while still doing right by your domain to make sure critical risk is considered right?

It's like that typical IT problem of, what do you mean you don't wanna approve that tool? But it's useful to me. Well, I know it is, but you know, stuff.

Robin Atwood: Oh my. Yeah. I approach AI as I approach anything else potentially disruptive, and I've seen a lot of this throughout my career. Anybody remember the code free software development line that everybody was pushing a while back?

Eddie Flaisler: that died quickly.

Robin Atwood: I approach it with cautious optimism. I [01:01:00] recommend studying what the different models excel at or they break down, and the overall design of each system. For example, AI can write excellent code or it can create horribly structured systems that seem on first glance to be workable, but that are full of bugs and other problems. In general, I recommend treating AI like an employee who is just getting started. Give it tasks, check up on them over time and you'll find out what they do best and what tasks to avoid assigning them. One thing it can do very well, at least certain models can, is analyze existing code. That can be very helpful when you're looking at rapidly changing systems where engineers haven't had the time to document or we're looking at best practices. As with any tool, it is vital to get to know it and how to use it properly. Giving your team a sandbox in which to play with it can be very helpful.

And as you gain knowledge and confidence in its abilities, encourage its use. One of the first use cases that made a huge impact on my team was proofreading. For team [01:02:00] members for whom English is a second, third, or maybe even fourth language, Having a system trained on our languages idiosyncrasies can be invaluable. Would I want to read a novel written by AI? Probably not. But notifying folks that a service will be taken down for upgrades at 10:00 PM? Of course, the biggest weakness so far that I can see, the proverbial achilles heel of ML is that so far there is no real persistence of experience or even knowledge of consequences. Perhaps AGI, artificial general intelligence, will solve that. Maybe not. I think there is something beautiful about the concept that while AI may be enormously capable of performing task, it takes a human to orchestrate those tasks and understand their utility. It will also take a human to understand the potential impact and orchestrate their use to the best efficacy.

Eddie Flaisler: 100%. In the last episode, Morgan and I were just talking about the issue of explainability in AI [01:03:00] and how difficult it is to measure something you do not understand fully. Right? How can you tell if it's gonna behave well in all cases? If you're not actually in the weeds of what's going on from the inside, and that's an inherent limitation to this technology. And it'll be just very interesting to see how different IT teams deal with that in the future.

Morgan VanDerLeest: Such a great perspective and really appreciate you joining us today, Robin, and to the listeners, if you enjoyed this, don't forget to share and subscribe on your podcast player of choice. We'd love to hear your feedback. Did anything resonate with you? Or more importantly, did we get anything completely wrong?

Let us know. Share your thoughts on today's conversation to peopledrivendevelopment, that's one word, peopledrivendevelopment@gmail.com, or you can find us on X or Twitter @pdd pod. Bye everyone.

Robin Atwood: Bye everyone. Thank you so much for tuning in.

Eddie Flaisler: Thank you for being with us, Robin. So long

​