owen

@Mary Poppendieck, I have a question to kick things off. You have been quoted as saying that the product owner is the worst thing that Scrum has visited upon the software world. What did you mean by that?

Mary Poppendieck

Hi Owen. For the decades before Scrum, it was the job of the technical team to truly understand the problem to be solved, engaging with customers as needed. This was real engineering - presented with a challenging problem, devise a solution that is efficient, creative, and reliable. This role was removed from software engineers - to some extend - by the business analyst, and to a much larger extend by the concept of projects, where project managers came up with solutions to problelms.

owen

Do you think there are any applications where a product owner can add value to a technical team?

Mary Poppendieck

In the last two decades, the idea that the technical team engineers a solution to challenging problems has largely disappeared, replaced by a 'product owner' who decides what the team should do and sets priorities. This role was part of the software engineering job in earlier times.

Tom Stiehm

In your opinion, what changed? I.e. why did we evolve this way?

Samer Adra

We could probably get quite a list of the negative things Scrum, or badly done Scrum, has inflicted.

PO and ScrumMaster roles (don't worry, the ScrumMaster will learn all the theory and solve your impediments for you)

Fixed timeboxes: Implicit lower limit on batch size and WIP; artificially constraining schedule and leading to pulling stories in, splitting stories, and moving stories to the next sprint (and then we'll have a retro talking about how we need to get better at planning/estimating/commitment/execution but with no discussion of "by what method" or root cause.

Overreliance on estimation, often treated as commitment, with silly things like story points, ideal hours, etc.

Overreliance on ceremonies to batch up all the planning, reviewing, kaizen, etc.

Distrust between management and doers, chickens vs. pigs

In general, prescriptions for how & when to do everything over understanding and theory

Certifications and scaling frameworks

Mary Poppendieck

Good list. When I wrote code, I was in an engineering department and we created control systems for massive roll-goods processes. When I compare how we worked to the Scrum formulas, I know we would have been deeply insulted by the juvenile ceremonies Scrum recommends. I think it's time we started treating everyone like adults.

---

Samer Adra

I can personally say it feels a lot more empowering as a team member when I'm actually involved in understanding what the customer wants for myself, and being part of decisions with less handoffs.

Mary Poppendieck

I am sure most engineers are energized by a challenge - and rising to meet that challenge with an elegant solution.

---

Samer Adra

@Mary Poppendieck how have you changed your views over time? For example, when your first book came out in 2003, Kanban wasn't a thing. And maybe it's just me, but your talk "the Elephant in the Room" (which I loved and wish everyone would watch) has a more Demingy/purist feel to it than your earlier work on the topic, which seem to give some wiggle room for individual performance appraisals/incentives.

Mary Poppendieck

Hi Semer,

Samer Adra

(meaning David Anderson style Kanban wasn't a thing in software, obviously not with regards to manufacturing.)

Mary Poppendieck

In my opinion, a generation is software is about 10 years - a decade. Anyone in the field who does not change the way they look at and talk about things every ten years of so is guaranteed to be out-of-date. In 2000, a three week iteration was very fast. Now it is incredibly slow. I just read about branching for a sprint. Branching has been largely replaced by trunk-based development; sprints by continuous delivery.

Mary Poppendieck

I still think appraisals are a really bad way to keep people aware of how they are doing, and if they are necessary, the leader is not doing her job. I think incentive pay is a huge disincentive and should never be used. But today's interesting topics seem to be more about software engineering

Mary Poppendieck

I like some form of kanban as a way to manage WIP and flow. It is, of course, not the only way to do this. For example, kanban boards were freat for team rooms - but do they work for remote teams? What I found is that with remote teams, the biggest problem can be getting rapid code reviews, and managing the code review process effectively is almost the same as managing WIP.

Samer Adra

I would love if more people got interested in the people an organizational aspect of things as well, because sometimes that can be a limiting factor. But it's true that there are exciting changes happening in the engineering space, and a lot of people are waking up to it and embracing devops. Thanks!

Samer Adra

True, I find code review is often a bottleneck. Seems pairing or mobbing would help (while reducing WIP), but the distributed/remote aspect makes that harder.

The people behind TameFlow seem to have a good understanding of things, but I haven't been able to dive in and learn it yet.

Mary Poppendieck

The model for remote teams is Open Source. and in open source, committers review code on a volunteer basis. So if you want your code reviewed, it had better be short and easily readable.

Mary Poppendieck

But also, someone has to care about reviewing and make it a top priority.

Samer Adra

Interesting about the FOSS model, especially with them all being volunteers. What I've seen happen when pull requests get too big is eventually it becomes urgent and someone is forced to look at it, resulting in a big debate about code design (often the people who make huge pull requests aren't going to have the cleanest code or the best testing), but we're forced to get the feature out and push it through, while agreeing to get to the feedback later (which might never happen).

---

Tom Stiehm

@Mary Poppendieck Given your PO response, what should a software team look like?

Mary Poppendieck

Hi Tom.

Mary Poppendieck

to answer your first question - what happened, why did we evolve away from a focus on software engineering - I think that the 'contract' model of software development took over IT departments in the 1990's: that is write a contract and outsource the work. Didn't matter if it was internal or external. I believe this happened when the IT department was treated as a cost center, so cost reduction was the most important thing to focus on.

Tom Stiehm

So true, thanks for the response.

Mary Poppendieck

This model does not respect that the problem to be solved is challenging, complex, and evolving. Maybe so when software was about automating underlying business processes. Not at all so when software is about developing products.

Mary Poppendieck

So what should a team look like? Well, what does a product engineering team look like of the product is softgoods (like tape at 3M) or hardware?

Mary Poppendieck

The product team is a part of the line business. It is NEVER in a cost center.

Mary Poppendieck

2. The product team has various line functions: at 3M a product team had tech, marketing, manufacturing, and quality as key (and equal) members.

Mary Poppendieck

3. The product team has a leader - at 3M we had a product champion - who cared deeply about the product's success, and who was responsible to make it successful - sort of like the CEO of a small business.

Mary Poppendieck

I like to think of the team as a small start-up business, and if you investigate AWS, that is exactly the way service teams are structured. AWS is an amazing organization that puts dozens of enterprise-level services on the market every year - year in and year out - and has been doing so since 2012. This is an amazing fete - and studying hwo they manage to do this is probably time better spent that checking out SAFe.

---

Samer Adra

@Mary Poppendieck what are some successful adoption patterns you've seen for organizational change? For example, one change I'm currently focused on is moving from separate dev/test silos with testing at the end to devops mindset and building quality in. Curious about things like top-down, grassroots, middle-led, start with theory vs. start with doing or show a small success, getting everyone on board, etc. (In my example I'm seeing lots of devs content to not even do unit testing and to let QA handle things, and testers are content to do everything after the fact manually or with UI automation.)

Mary Poppendieck

Let's see - since I write books instead of code these days (the two are not all that different), I have to start with my go-to word processor, which is Microsoft word. Our home is automated, and I program it through the ISY software iand portal.

Mary Poppendieck

I use most Office products, Quicken and QuickBooks, and Adobe Premiere and Photoshop. I like to use as high a level tool as is available.

Mary Poppendieck

Opps, this is in the wrong thread!

Mary Poppendieck

Now to answer your question:

Samer Adra

Mary Poppendieck

I would suggest you are thinking too narrowly. Organizational change - especially when it is about improving digital technology - should start at the overall business level, not the technology level.

Mary Poppendieck

There is a big difference between an agile transformation and a digital transformation. Agile transformations are about improving technology. Digital transformations are about changing the organization so it can compete more effectively in the digital world.

Mary Poppendieck

It's not a question of top-down or bottom up. It's a question of WHY?

Samer Adra

I'm probably thinking that way because of my limited influence and authority as a developer. I recently read "Corporate Rebels," which was all about ways to restructure business, but it seems like an insurmountable challenge for someone without authority at a company of thousands.

Mary Poppendieck

For what reason is thje organizational change necessary? Today, many large businesses are threatened by smaller businesses that attack a small portion of their revenue stream with digital technology.

Samer Adra

So even though I know it's obvious that we need to break down barriers, build quality in, optimize the whole, etc., the "why" question needs to be framed better to (hate this term) "the business."

Mary Poppendieck

In this situation, the response has to be to understand the comparative threat and respond to it, That is not a technology improvement issue, it is a business leadership issue.

Samer Adra

We're already doing all the tech changey things like cloud migration and AI/ML. We have a central "Lean-Agile" team so people think that checks the box.

Mary Poppendieck

Lean is first and foremost about delivering more value to customers. So that is the question to ask: are we delivering the best value to our customers from their perspective?

Mary Poppendieck

You have to see the world as your customer sees it. What could you be doing better/differently to make your customers even happier than they are now? What customer problem are you solving and could you solve it better?

Mary Poppendieck

Can I ask what domain you work inn?

Samer Adra

I work in devtools so one thing I'm doing is building dashboards with test/coverage and static analysis results of all our repos. I'm hoping those results will scare people. Also working to measure and graph the four metrics from the Accelerate book (delivery lead time, change failure rate, MTTR, and release frequency). Those are things the business sees value in and would want to optimize.

Still a jump from there to get people to understand a lot of the theory, but hoping to at least wake people up to the need.

Samer Adra

Investment and money management such as 401(k)

Mary Poppendieck

Ah ha. This is a field seriously under attack from Fintechs.

Samer Adra

Still dealing with mandatory change control policies with approvals and 2-day waiting for any deployment, but we have agreement that we can get instant automated approval if we can prove quality is above some gate. I know not everything is measurable but that's where we are now.

Mary Poppendieck

And you mentioned the telling term "the business". Which means you are not part of a business team, they are 'other'. Am I right?

Samer Adra

yeah a lot of discussion about how to get alpha, getting better returns without relying entirely on human analysts

Mary Poppendieck

Do you know what your CEO looses sleep over at night?

Mary Poppendieck

I would guess that your various revenue streams are under atack, revenue is flat or declining because fees have to be lowered to stay competitive.

Samer Adra

no I'm not in "the business" and I haven't met the CEO to really know. but I imagine nowadays it's things like coronavirus and sovereign debt.

Samer Adra

absolutely right. even though we have consistently better rated funds and better returns (we do active management more than passive/index funds), a lot of people choose based on fees, and we we have recently lowered our fees to compete

Mary Poppendieck

so yes, coronavirus, but CEO's have to see out a few years - and just about every financial services organization in the world is worried about declining revenue due to digital techniques that lower prices - techniques that old companies have a devil of a time delivering, but Fin techs o it easily.

Mary Poppendieck

lowered fees in order to compete, but not lowered cost structures, I'll wager.

Mary Poppendieck

And THAT is what keeps your CEO up at night/

Mary Poppendieck

While it is obvious to you what tech might do to meet this competitive threat, it is hardly obvious to your business's senior management.

Samer Adra

yes that would be a big problem. so we need to eliminate waste, right? but we have a "Lean Agile" group on the software side and we have a "Business Process Improvement" group that also talks about (probably bastardized) Lean on the business side, and I think that might be a limiting factor, as improvement has been delegated to those groups.

Mary Poppendieck

Because if it were, you would not be in an IT department, you would be in a business unit helping strategize about how technology could be combined with your business expertise to blow away the competition - not merely lower fees.

Mary Poppendieck

Business Process Improvement - as if business process is what is causing you to have to lower fees and yet you can't decrease cost structures?

Samer Adra

Deming would refuse to consult with a company unless the CEO met with him to lead the effort. but I feel like I'm not in a position to do that.

Mary Poppendieck

Right, but spend time understanding where your management is coming from - it can't hurt!

Samer Adra

certainly not. thanks!

Mary Poppendieck

So revenue is flat or declining, and technology is a cost center. Therefore, the money available to fund the cost center is declining (you are no doubt funded as a % of revenue).

Mary Poppendieck

So cost pressure is tremendous - but lower costs will not begin to address the underlying problem. Your organizational structure is not appropriate for competition in a digital age.

Samer Adra

revenue has been great even with lower fees because lower fees attract investors, and the market's growth in 2019 meant there was plenty of money. obviously that has now changed.

Mary Poppendieck

In a competitive battle, a profit center will beat a cost center. Every. Time.

Mary Poppendieck

And some financial institution management gets this. They are reorganizing into business teams chartered with addressing a competitive market. These teams need tech people badly, as well as savvy folks who understand the financial marketplace.

Mary Poppendieck

Your tech tools and business process improvement efforts are probably important, but they are unlikely to address the underlying change in the financial marketplace.

Mary Poppendieck

Insurance companies have a similar problem - the management of risk is what they do, but in the age of AI and IoT, risk management is a whole new game. Some companies will get this and thrive. May will not.

Samer Adra

A lot to think about. Really it seems like the improvements I'm trying to make in tech like measuring code quality, unit testing, etc. might be... I don't want to say suboptimizations because I think there is benefit to the whole... but the bigger wins are from higher-level organizational changes

Mary Poppendieck

Well, you really do have to lower costs, and what you are doing will certainly help.

Mary Poppendieck

Whether or not they are sub-optimizations or simply small steps in the right direction depends on your attitude. Are you scanning the horizon for changes in financial services that might make a big difference in your world? are you looking at your cost structures and trying to find the biggest problem. Are you mapping the flow of your customer-facing services and looking for the biggest bottleneck?

Mary Poppendieck

Do you even know who your customers are and what they care about most?

Mary Poppendieck

These would be good things for you to read up on at night.

Mary Poppendieck

Good questions to bring up at team meetings.

Mary Poppendieck

Above all, do not assume that someone else knows what the next best thing to work on is - they probably don't. Get involved in the conversation about what the team should focus on next.

Samer Adra

I am a customer of my company, trying to save for retirement and seeing the crazy markets. we have a good story for when times our rough that our active investors typically produce better (or less bad) returns than the market. But these are all good questions, and certainly in my role I tend to stay toward the tech side. Even the tech team I work on, in developer tools, is not even a gemba -- I work on tools and infrastructure that support the gemba such as helping them move toward CD. So some level of separation from the customer there.

Samer Adra

when times our rough => "when times are rough"

Mary Poppendieck

Yep, except that you ARE your own customer. And the markets are crashing as we type.

Mary Poppendieck

So if cost pressures were bad last week, they will be a lot worse going forward.

Mary Poppendieck

Tough times often offer opportunities for rethinking business-as-usual.

---

Shak

@Mary Poppendieck what software do you prefer, and why? (Any software which can be used for agile projects, for example Jira, Sifter, Trello, spreadsheets, emails) (edited)

Samer Adra

See Mary's responses in the thread I started just before your question.

Shak

Cool thanks @Samer Adra I see her response here: https://techwellhub.slack.com/archives/CDKPF4UBA/p1584024309047000?thread_ts=1584024095.045600&cid=CDKPF4UBA

Mary Poppendieck

Let's see - since I write books instead of code these days (the two are not all that different), I have to start with my go-to word processor, which is Microsoft word. Our home is automated, and I program it through the ISY software iand portal.

From a thread in #agile | Yesterday at 10:45 AM | View reply

Shak

and here: https://techwellhub.slack.com/archives/CDKPF4UBA/p1584024428047600?thread_ts=1584024095.045600&cid=CDKPF4UBA

Mary Poppendieck

I use most Office products, Quicken and QuickBooks, and Adobe Premiere and Photoshop. I like to use as high a level tool as is available.

From a thread in #agile | Yesterday at 10:47 AM | View reply

Shak

Also thank you, @Mary Poppendieck

---

Kelly M

@Mary Poppendieck I hear you were a software engineer in the 1960’s, 70’s and 80’s. What was coding like back then?

Mary Poppendieck

Hi Kelly,

Mary Poppendieck

in the 60's, most coding was done in assembly language. Note that many of the waterfall practices stemmed from the fact that code was significantly more difficult to debug at this level. On the other hand, there were no intermediaries between 'programmers' and the people/machines who used the code.

Mary Poppendieck

In the 70's, we began to see higher and higher level languages.

Mary Poppendieck

By the 80's, we had PC's (with the first apps on top of an operating system) and 4th generation languages. It was thought that programmers would no longer be necessary.

Mary Poppendieck

In the 70's, minicomputers replaced mainframes, and software began to be used in engineering as well as for business process automation. I worked in engineering computing, which was nght and day different from software done in the IT department.

Tom Stiehm

Funny, we are still looking for ways to make programmers no longer necessary. Whether is it low-code or AI. What do you think drives us to focus on finding silver bullets?

Mary Poppendieck

I worked in an engineering department that designed and installed process control systems for 3M roll goods processes. This was an internal engineering department. When I arrived, there was a big effort to replace wires and discrete components with minicomputers.

Mary Poppendieck

The effort to replace programmers in the 80's failed. The complexity simple rose to a higher level before software engineering became critical. That will always be ture.

Mary Poppendieck

Silver bullets are chased by people who do not understand the details of the problem but are desperate for a solution. Not appreciating the challenges (and not wanting to accept the difficulties), people look for the solution that is simple, cheap, and wrong.

Mary Poppendieck

We see a lot of this, all around us, every day, certainly not just in software. Human nature, I figure.

Mary Poppendieck

There is a big difference between long range thinking and short range thinking. I am biased toward long range thinking - it is the classic Lean approach. However, most western cultures reward short range thinking.

Jordan Allain

thank you so much for sharing!

Kelly M

Oh wow. Yes thanks so much for sharing!! Sorry had gotten pulled into a meeting.

---

Tom Stiehm

@Mary Poppendieck Great discussion about adoption patterns, how do you go about getting business leaders to buy into integrating the sperate silos of dev, ops, and business to actually work together? I have seen some leaders get out of the software as a cost center trap but others hold on to as hard as they can.

Mary Poppendieck

Structural change is very, very difficult. Lou Gerstner, the person who turned around IBM, said that the most difficult thing was to change the culture of people who had been sheltered form market realities by an extremely successful company. Turning around the company was like taking a lion raised in captivity and teaching it how to survive in the wild.

Tom Stiehm

Great analogy, I have found many times that the more successful a company is or was, the harder it is to get their employees or alumni to understand different circumstance and to react to that.

owen

Love the analogy

---

Tom Stiehm

@Mary Poppendieck Do you have any thoughts on how much design should be done upfront? Some agilest say no design up front only emergent design, some say some design up front, and from the middle and heavy weight methods we get big design upfront. I generally find some design up front with most of the design details being decided as you go works well.

Mary Poppendieck

Good question. First and foremost, remember that software in ALMOST ALWAYS part of something else, and that something else is highly variable - from a web site to a moon shot. The design question is about the larger context, not about the software.

Mary Poppendieck

Second, let's take SpaceX for an example. there is a wonereful youtube video https://www.youtube.com/watch?v=bvim4rsNHkQ which shows boosters blowing up on landing.

YouTubeYouTube | SpaceX

How Not to Land an Orbital Rocket Booster

Mary Poppendieck

Hard to see a government funding such an engineering approach, but it was FAR cheaper for SpaceX to blow up a few boosters than to figure out everything up front.

Mary Poppendieck

Check out these numbers:

Mary Poppendieck

Cost to launch 1 kilo into space:

1.Space shuttle: $54,500

2.1970-2000: $18,500

3.SpaceX: $2,720

NASA cost per launch $152 million

SpaceX $2 million – 1.3% of NASA Cost

Mary Poppendieck

The program to develop a reusable booster was FAR CHEAPER because the boosters did not have to be perfedt upon test launch.

Mary Poppendieck

This is CLASSIC ENGINEERING. it is not new, and not particularly related to software. Toyota uses a technique called set-based engineering that is similar.

Mary Poppendieck

Now, if people are in the spaceship, you do not want it top blow up. But sometimes the best way to ensure that does not happen is to find the limits by letting it blow up a few times during development.

Tom Stiehm

Great perspective and thanks for the opportunity to ask you some questions, I get a lot out of your answers.

Mary Poppendieck

I really wish that we would take a look at how good engineers address the problems we have in software; they've been at it for a few more decades.

Mary Poppendieck

Let me give one last example. I program my home light control system. I do not do test-driven development. I do not do much of any design. I just try things that strike me as interesting and see how they work out. There are no consequences for failure, so why worry about it?

---

Samer Adra

@Mary Poppendieck do you think the DevOps movement is heading in the right direction, or do you have any concerns?

Samer Adra

I've encountered some people who seem to think DevOps is just the newer way of doing ops (and is somehow still separate from dev).

Mary Poppendieck

Well, remember that DevOps is over 10 years old, so you have to ask the question - how have things changed since 2010? In 2010, the cloud was not something big companies considered as an option. Today, given the cost pressure on all IT departments, not considering cloud alternatives can be irresponsible.

Mary Poppendieck

So is DevOps in a cloud world the same as DevOps in its original incarnation? maybe and maybe not

Mary Poppendieck

If DevOps means get Ops people on the Dev team or get Ops and Dev working together - well, that's table stakes. No longer a big thing.

Joshua Hillerup

I think if you're in a context where you're not able to do cloud (or it's one of the rare situations where it actually isn't what you should use for everything) you can still certainly do devops

Mary Poppendieck

If DevOps means developing a very viable delivery and rollback pipeline, that's a lot of work and needs a lot of attention.

Mary Poppendieck

But - question - where does product fit in to this picture? I've heard the term dev-ops-product. good idea to think more about the whole team, not just dev and ops.

Mary Poppendieck

and if there is a cloud, is ops the same as it was?

Mary Poppendieck

and what about SRE's? are they the new Ops? and where do they fit?

Joshua Hillerup

so, you can make an argument about what devops "is", but the problem it's trying to accomplish are those that come from manual intervention in things

Samer Adra

I feel like DevOps is an umbrella term that includes Lean and Agile ideas. Some of books on it talk about Deming and Goldratt. To me a big part of it is mindset, breaking down barriers, CD, stuff that’s all valid in the cloud

Mary Poppendieck

Hey Josh, I agree, you cannot always use a cloud. But still, the cloud has changed the way we think about product teams.

Joshua Hillerup

people are good at some things, computers are good at others. Make the computers do everything they are good at, and leave the humans for the tasks that humans are better at.

Joshua Hillerup

specifically for DevOps in the case of IT operations

Joshua Hillerup

And yeah, the cloud did massively change that, but a lot of the learnings can be very useful for on prem pieces as well

Mary Poppendieck

Hi Samer. I have seen DevOps used as an umbrella term. I am a bit concerned because the name specifically excludes other disciplines.

Samer Adra

Devops isn’t a great term because it should be DevSecTestProductOps

Mary Poppendieck

No - won't stick and STILL iexcludes - for example - SRE's.

Joshua Hillerup

Yeah, the name sucks. Although the fact it turns a lot of ops into requiring dev skills does kind of fit a bit.

Mary Poppendieck

My fallback is software engineering.

Mary Poppendieck

Because then whatever the big thing of the day is (eg. today it's AI) it is included.

Mary Poppendieck

I see artificial intelligence as a big deal in the next decade. So the questions that come to mind are:

Mary Poppendieck

how does AI affect software engineering and product development?

Mary Poppendieck

2. What responsibility do software engineers have for the safety of the products their software controls?

Mary Poppendieck

I programmed process control systems - systems that could kill people. We were held responsible for the safety of the systems we design.

Mary Poppendieck

If you were to write code for a vehicle automation system, what is your responsibility for the safety of the vehicle's passangers?

Mary Poppendieck

If you work on an SI system that creates a bias through a limited training set, what is your responsibility?

Mary Poppendieck

AI system

Mary Poppendieck

I think these are big questions for the entire team working on such products, and the team should be structured to have the debate.

Joshua Hillerup

I think this all comes down to software verification, not AI exactly

Joshua Hillerup

Like, software verification is important for AI, but the role of AI in helping with software verification so far has been limited

Joshua Hillerup

But for devops the advantages are huge

Joshua Hillerup

Although certainly they don't solve all the problems

Mary Poppendieck

Yes, I agree. But I do not think all systems can be completely verified nor should they necessarily be.

Mary Poppendieck

When I did process control systems, we held these things to be true:

Mary Poppendieck

all systems that innvolve hardware will eventually fail at some time.

Mary Poppendieck

2. It is impossible to guess all failure modes, although it is required to try to guess.

Mary Poppendieck

3. because at some time every system will fail in an unexpected way, designing fail-safe systems is mandatory.

Mary Poppendieck

So robust safety meant assuming failure and designing resilience and fail safe modes into every system.

Mary Poppendieck

Verification is an economic, not a safety consideration.

Mary Poppendieck

Failure handling is a safety, not an economic consideration.

Mary Poppendieck

At no time should economics interfere with the design of safe systems. But how much failure can be tolerated and still have a safe system is an economic consideration. For example, my lighting system can fail with no consequences - so no verification needed.

Mary Poppendieck

My question for product teams is this: is the entire team (no matter what you call it: dev-ops=prod-design-... in agreement on the issues which cannot be compromised, and on those, does everyone have a say?

Joshua Hillerup

I've never been in a situation where the entire team as a whole for a product agreed on anything like that

Joshua Hillerup

It's always been much more hiarchical

---

Samer Adra

Any upcoming writing? Really appreciate your Lean Software Development books and lean essays.

Mary Poppendieck

I do not intend to write another book, and have not written any essays recently, but that may change now that I am stuck at home for some weeks - like just about everyone else in the world. :face_with_raised_eyebrow:

---

Tom Stiehm

@Mary Poppendieck you mentioned AI having a big impact in the next ten years. What do you mean? While we are always getting better, I still feel most discussion of AI is similar to expecting magic to happen. Is your view or experience where can AI help?

Mary Poppendieck

The big breakthrough in AI happened in 2012. Now I see articles questioning if AI will change the way we write software - will software even be necessary. Well, engineering thinking will always be necessary, but the nature of our learning feedback loops will change.

Mary Poppendieck

Don't forget that the technical enabler of agile was our ability to test automatically, rather than manually. So we found ways do do this testing more and more effectively to the point where we now routinely use automated deployment pipelines with automated roll-back.

Mary Poppendieck

Now AI brings a new and extremely effective variable into the feedback loop, and those who do not learn how to capitalize on it will be left behind. At least, that's the way I see it.

Mary Poppendieck

Engineering is all about developing effective learning / feedback loops. SpaceX is so cost-effective because they totally changed the way rockets are developed through a whole new approach to learning.

Mary Poppendieck

Tesla is changing the feedback equation for automated vehicle control. Tensor flow allows 'non-programmers' to develop very useful feedback loops.

Mary Poppendieck

Teams that are still branching for two weeks and then merging to the trunk are at least two generations behind in using the latest technology for learning/feedback loops.

Mary Poppendieck

Really, I think AI is the continuous delivery of the 2010's.

Tom Stiehm

Wow, that is the most cogent argument for AI in software development I have heard. Glad I asked.

Mary Poppendieck

I have been in this software world for over half a century, and if I've learned one thing, it's that anything that you have to keep on top of the future, not get stuck in the past. The rule of thumb is that if you are working with 10-year-old ideas, it's time to look around. there is something out there that is going to bite you - unless you find it first.

Tom Stiehm

That is so true, many of the organizations I work with on transformation have their practices stuck at best practices 10, 15, or more years ago. Often the first part is just brining them up to speed on what the industry has been doing since they stopped learning. (edited)

Mary Poppendieck

You have to work from a set of underlying principles: customers first, flow is fundamental, feedback (learning) is essential, quality (reliability, safety, etc) must be built in. But what that means is forever advancing and you have to keep pace with the latest and greatest ways to assert the fundamental principles into your work.

---

Samer Adra

@Mary Poppendieck do you think statistical process control has any applicability in SW development? Understanding variation is very useful when counting defective widgets but seems tricky in knowledge work. But without it there is the risk of using the most two recent numbers (of anything, lead time for example) and saying we got better or there is a problem.

Mary Poppendieck

I have seen so much damage from attempts to suppress variation when in fact, variation is necessary for learning.

Mary Poppendieck

There is nothing wrong with Statistical process control - we used it aggressively when qualifying our new process control lines at 3M. This was BEFORE it was popularized by Deming.

Samer Adra

The same is said about the chaos quadrant of Cynefin. Doesn't sound like a good place to be in, but it's said that chaos is where a lot of innovation happens.

Mary Poppendieck

But at the same time, we had a big program investigating neural networks for determining how to run the most efficient controlled experiments when qualifying a line. Yet. Neural networks were a thing in the 80's. But the compute power did not exist to really leverage them. It does now.

Samer Adra

Deming was teaching it to Japan in the '50s and doing it in America before that, but certainly there was a period in America before the '80s where apparently people forgot about it

Mary Poppendieck

Not everyone forgot. I learned it in my engineering department and it was in common use in our manufacturing plants.

Samer Adra

Wondering if we're charting things in SW, lead time, change fail rate, etc., should we be concerned about trying to identify if a change is common cause vs. special cause.

Mary Poppendieck

Hmmm. If it's useful, yes. if there are more useful ways to get a handle on what's going on, then no.

Mary Poppendieck

I am concerned about the words you use "failure rate"

Samer Adra

Jira has a graph that it calls a control chart, but the upper/lower limits are a rolling +- 1 STDDEV, so naturally there are a lot of "outliers" and it's quite sad.

Samer Adra

% of releases that result in a customer incident

Samer Adra

is that not something we want to minimize? maybe not at facebook where they "move fast and break things"

Mary Poppendieck

Upper and lower limits of +/- 1 standard deviation means you have accepted a certain failure rate as 'normal'.

Mary Poppendieck

1 sigma means 68% good. Why would that be the right control limit?

Mary Poppendieck

2 sigma is 95% good. Is 95% appropriate for the situation or not?

Samer Adra

it's not at all. normally in p charts it's 3. and in xMR charts it's not even STDDEV but based on range. I just opened Jira the other day and saw their build-in "control chart" and was shocked that it was 1 STDDEV

Mary Poppendieck

shocking indeed

Mary Poppendieck

but the situation dictates what level of "failure" is acceptable.

Mary Poppendieck

Control charts are nice things to use with Kanban - you can chart the throughput of a task (or story or feature or defect or whatever) from accepted to completed.

Samer Adra

talking about sigmas reminds me of six sigma. I wonder if the term "Lean" has poisoned by 6σ (and also people who love to copy tools).

Mary Poppendieck

Then you can do a control limit of - say - 2 sigma - and find out how long it typically takes (95% of the time) for something to get through your system.

Mary Poppendieck

That control line is what you can promise-date for delivery time, and 95% of the time you will be within your promise date.

Mary Poppendieck

Viola! No estimating needed and much more reliable promise dates.

Samer Adra

I've read about mature Kanban teams doing that and having SLAs that give confidence a story will be done in a certain amount of time. unfortunately I'm still trying to fight people about the issues of Scrum. :rolling_on_the_floor_laughing:

Samer Adra

Mary Poppendieck endorses #noestimates!

Mary Poppendieck

I don't like 6 sigma - because it does not focus on the important things typically.

Mary Poppendieck

Nor di I have much use for Scrum. But I don't fight it, I just offer alternatives.

Samer Adra

I've seen people who still treat a sprint plan as a commitment instead of a forecast. Seems like that kind of metrics-based SLA would be useful for them

Mary Poppendieck

To be clear, I offered a far better way to estimate deliveries than the typical way.

Samer Adra

right. forecasting over guessing

Mary Poppendieck

I am so far beyond the concept of iterations that I hardly think about them any more. Those who are still doing iterations are 2 generations out of date, and they have bigger problems than considering a sprint a commitment.

Mary Poppendieck

Meteorologists forecast based on evidence. A few people can look at the sky and guess ahead a day or two, but that's about it.

Mary Poppendieck

The reason Kanban is still relevant is because it encourages people to thing about WIP and FLOW. Those two things never go out of date, although the method for achieving them does. For example - pretty much all of our colleagues are going to be working from home for a while. Thus no face-to-face meetings, no physical kanban charts. But WIP and FLOW remain seriously important - so how to achieve them when everyone is working from home? Adopting the practices from an office workspace is unlikely to be the best answer.

Mary Poppendieck

One of the bigger SLA's today is the reliability of a data center or cloud environment. Here you can be sure that mean time between failure, mean time to repair, and breadth of damage are key and are measured based on history.

Mary Poppendieck

We absolutely can do that in software - for any item moving through our system, and for any deployment, we can measure and improve those things.

Mary Poppendieck

So why would we not to that instead of these last century techniques that are basically useless?

---