-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathoperations_iwc.html
456 lines (421 loc) · 20.7 KB
/
operations_iwc.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Operation Re-Think</title>
<meta name="description" content="A set of ideas around r-thinking and re-shaping operations">
<meta name="author" content="Martin Sumner">
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, minimal-ui">
<link rel="stylesheet" href="css/reveal.css">
<link rel="stylesheet" href="css/theme/black.css" id="theme">
<!-- Code syntax highlighting -->
<link rel="stylesheet" href="lib/css/zenburn.css">
<!-- Printing and PDF exports -->
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? 'css/print/pdf.css' : 'css/print/paper.css';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
<!--[if lt IE 9]>
<script src="lib/js/html5shiv.js"></script>
<![endif]-->
</head>
<body>
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section>
<h2>Operation Re-think</h2>
</section>
<section>
<h2>Today we will cover ....</h2>
<p align="left">A very brief retrospective on Agile</p>
<p align="left">Discuss the changing landscape of operations</p>
<p align="left">Look through the perspective of real world problems</p>
<p align="left">Look at the success of some real world answers</p>
<p></p>
<p><small>http://martinsumner.github.io/presentations/operations_iwc.html#/</small></p>
</section>
<section data-background="#A3C2FF">
<h2>What Did We Learn From Agile?</h2>
</section>
<section data-background="#A3C2FF">
<h2>In the beginning there was an Act of Rebellion</h2>
<p align="left">The manifesto didn't just state what we would do ...</p>
<p align="left">... it stated the negative practices we would no longer do</p>
<p align="left">There was a view of normal... </p>
<p align="left">... which people sensed wasn't working</p>
</section>
<section data-background="#A3C2FF">
<h2>Someone noticed humans were involved - and they may not be perfect</h2>
<p align="left">Lake wobegon principle</p>
<p align="left">Hindsight bias</p>
<p align="left">Confirmation bias</p>
<p align="left">Dissonance reduction</p>
<p align="left">Sunk cost effects</p>
<p align="left">Zero-risk bias</p>
</section>
<!--<section data-background="#A3C2FF">
<h2>Conventional appeared rational because of cognitive biases</h2>
<p align="left">Applying scientific method showed ...</p>
<p align="left">... it wasn't rational to believe we could predict the future</p>
<p align="left">... if we predicted the future we wouldn't be rational about it</p>
<p align="left">The change to Agile was combatting our biases</p>
<p align="left">... Transitioning from believing in predictions to embracing uncertainty</p>
</section> -->
<section data-background="#A3C2FF">
<h2>The People Problem</h2>
<blockquote cite="http://hintjens.com/blog:125">
<p>“And so gradually I shifted from technical architect to social architect. From caring about technical designs to caring about people and the psychology that drives them. Because in the end, this is what seems to make the difference between a working project and a failure. ”</p>
</blockquote>
<p><small>Peter Hintjens, 2016</small></p>
</section>
<section data-background="#A3C2FF">
<h2>The Other Perspective</h2>
<blockquote cite="https://en.wikipedia.org/wiki/Scientific_management">
<p>“Now one of the very first requirements for a man who is fit to handle pig iron as a regular occupation is that he shall be so stupid and so phlegmatic that he more nearly resembles in his mental make-up the ox than any other type... ”</p>
</blockquote>
<p><small>Frederick Taylor</small></p>
</section>
<section data-background="#A3C2FF">
<h2>The Other Perspective</h2>
<blockquote cite="https://en.wikipedia.org/wiki/Scientific_management">
<p>“... The man who is mentally alert and intelligent is for this very reason entirely unsuited to what would, for him, be the grinding monotony of work of this character. Therefore the workman who is best suited to handling pig iron is unable to understand the real science of doing this class of work. ”</p>
</blockquote>
<p><small>Frederick Taylor</small></p>
</section>
<section data-background="#A3C2FF">
<h2>A parallel in history is detected</h2>
<p align="left">From Scientific Management -> to Lean, Deming, Ohno et al</p>
<p align="left">There was a lot to learn from looking at the parallel</p>
<p align="left">... about why it will work</p>
<p align="left">... and why it won't work for many</p>
</section>
<section data-background="#A3C2FF">
<h2>But what about Operations</h2>
<p align="left">err ... DevOps ... YBYRI ... Continuous Delivery </p>
<p align="left">... Is this just about suporting the delivery of the product?</p>
<p align="left"></p>
<p align="left">Do we need a rebellion?</p>
<p align="left">Have we got a system that doesn't understand humans?</p>
<p align="left">Will the hand of history guide us on the journey?</p>
</section>
<section>
<h2>The Changing Landscape Of Operations</h2>
</section>
<section>
<h2>Over to you ...</h2>
<p align="left">Imagine a timeline ...</p>
<p align="left">... For me it would be 1994 - 2016</p>
<p align="left">Think through the trajectory of Operations on that timeline</p>
<p align="left">Availability - Acceptability of Planned/Unplanned downtime?</p>
<p align="left">Security - Privacy? Detection or Protection?</p>
<p align="left">Cost - Running, Build, Opportunity?</p>
<p align="left">Performance - per Request or per Business Process?</p>
</section>
<section>
<h2> ... </h2>
</section>
<section>
<blockquote >
<p>“ People told me technology changes would make things easier and give me more leisure time, but technology changes just mean customers demand things sooner and understand failure less ”</p>
</blockquote>
<p><small>Anon</small></p>
</section>
<section>
<blockquote >
<p>“ Item #1 for the agenda should be "what are you going to do when Spine fails?" ”</p>
</blockquote>
<p><small>Anon</small></p>
</section>
<section>
<blockquote >
<p>“ In a real business situation, many organizations will throw up their hands and say, we will wait for a complaint and do a manual adjustment. Airlines are a case in point. If an airline system loses a reservation ... this will be detected when the passenger attempts to check in. All airlines have procedures to handle this kind of problem ... It is therefore not worthwhile to implement complex software processes to guarantee no loss of records. ”</p>
</blockquote>
<p><small>IT Architectures and Middleware, 2004</small></p>
</section>
<section>
<h2>BA Apologises</h2>
<img width="600" height="600" data-src="images/ba_apologises.png" alt="BA Apologises"/>
</section>
<section>
<h2>Google Apologises</h2>
<img width="600" height="300" data-src="images/google_apologises.png" alt="Google Apologises"/>
</section>
<section>
<h2>Never Fear - ITIL To The Rescue!</h2>
<blockquote cite="https://en.wikipedia.org/wiki/ITIL">
<p>“ After the initial publication in 1989–96, the number of books quickly grew within ITIL v1 to more than 30 volumes. ”</p>
</blockquote>
<p><small>https://en.wikipedia.org/wiki/ITIL</small></p>
</section>
<section>
<h2>Never Fear - ITIL To The Rescue!</h2>
<blockquote cite="https://en.wikipedia.org/wiki/ITIL">
<p>“ In 2000/2001, to make ITIL more accessible (and affordable), ITIL v2 consolidated the publications into nine logical "sets" that grouped related process-guidelines to match different aspects of IT management, applications and services. The Service Management sets (Service Support and Service Delivery) were by far the most widely used ... ”</p>
</blockquote>
<p><small>https://en.wikipedia.org/wiki/ITIL</small></p>
</section>
<section>
<h2>Never Fear - ITIL To The Rescue!</h2>
<blockquote cite="https://en.wikipedia.org/wiki/ITIL">
<p>“ ITIL 2007 edition (previously known as version 3) is an extension of ITIL v2 and fully replaced it following the completion of the withdrawal period on 30 June 2011.[4] ITIL 2007 provides a more holistic perspective on the full life cycle of services ... Most of the v2 activities remained untouched in 2007, but some significant changes in terminology were introduced ... ”</p>
</blockquote>
<p><small>https://en.wikipedia.org/wiki/ITIL</small></p>
</section>
<section>
<h2>Some thoughts</h2>
<p align="left">The world in which we manage services has changed</p>
<p align="left">We're armed with a set of service design ideas from another era</p>
<p align="left">... An era from which we've found the delivery ideas were broken</p>
</section>
<section>
<h2>Over to you ...</h2>
<p align="left">What represents best practice for Service Management?</p>
<p align="left">What represents conventional wisdom of how we should be doing this?</p>
<p align="left">What are you being told?</p>
</section>
<section>
<h2> ... </h2>
</section>
<section data-background="#A3C2FF">
<h2>Real World Problems</h2>
</section>
<section data-background="#A3C2FF">
<h2>Overview</h2>
<p align="left">Focus on problems with very very long resolution times</p>
<p align="left">People aren't stupid</p>
<p align="left">People may do things that end up looking stupid</p>
<p align="left">Think about the technology or the people - maybe both</p>
</section>
<section data-background="#A3C2FF">
<h2>Internet access has been unusable for 18 days</h2>
</section>
<section data-background="#A3C2FF">
<h2>Internet access has been unusable for 18 days</h2>
<p align="left">The guess propogated and perpetuated</p>
<p align="left">The guess was convincing</p>
<p align="left">Defining the problem was the first step</p>
<p align="left">Root cause in the investigation - and surely innocuous</p>
</section>
<section data-background="#A3C2FF">
<h2>A little bit of knowledge goes the wrong way</h2>
</section>
<section data-background="#A3C2FF">
<h2>A little bit of knowledge goes the wrong way</h2>
<p align="left">Faith in the first assumption was unshakeable</p>
<p align="left">Only the ignorant prepared to provide leadership</p>
</section>
<section data-background="#A3C2FF">
<h2>A triangle of blame</h2>
</section>
<section data-background="#A3C2FF">
<h2>A triangle of blame</h2>
<p align="left">The eventual fix was highly inefficient</p>
<p align="left">Behaviour was unhelpful, but resolved by a change in awareness not attitude</p>
</section>
<section data-background="#A3C2FF">
<h2>... 23, 24, 25, 26, 100</h2>
</section>
<section data-background="#A3C2FF">
<h2>... 23, 24, 25, 26, 100</h2>
<p align="left">It isn't anything like linear</p>
<p align="left">How hard a problem, to know the factors, and translate the outcome</p>
<p align="left">Ending Moore's Law means this is a sign of things to come</p>
</section>
<section data-background="#A3C2FF">
<h2>Its fairly slow at 9am, worse at 11am, becomes unusable after 2pm</h2>
</section>
<section data-background="#A3C2FF">
<h2>Its fairly slow at 9am, worse at 11am, becomes unusable after 2pm</h2>
<p align="left">This had been going on for 3 months</p>
<p align="left">The user spotted the pattern, the interpretation was too literal</p>
<p align="left">The guilty functionality came for "free", but monitoring it wouldn't be free</p>
<p align="left">Earlier optimisation guesses clouded the picture</p>
</section>
<section data-background="#A3C2FF">
<h2>Sometimes we get logged out when submitting a form, especially John</h2>
</section>
<section data-background="#A3C2FF">
<h2>Sometimes we get logged out when submitting a form, especially John</h2>
<p align="left">Meeting the user gave the key insight</p>
<p align="left">PICNIC? E-mails cannot be sent more than 500 miles</p>
<p align="left">PICNIC? Password cannot be entered standing up, only sitting down</p>
</section>
<section data-background="#A3C2FF">
<h2>The weekend war room</h2>
</section>
<section data-background="#A3C2FF">
<h2>The weekend war room</h2>
<p align="left">There were no communication barriers ...</p>
<p align="left">... But still there was no resolution to a problem falling between clean boundaries</p>
</section>
<section data-background="#A3C2FF">
<h2>Why is Spine II faster?</h2>
</section>
<section data-background="#A3C2FF">
<h2>Why is Spine II faster?</h2>
<p align="left">Lots of magic:</p>
<p align="left">NoSQL, Moore's Law, open source, Agile, commodity hardware</p>
<p align="left">Point 2 of a second - people always cared about this ... </p>
<p align="left">... and there was magic before</p>
</section>
<section data-background="#A3C2FF">
<h2>Why is Spine II faster?</h2>
<p align="left">Two stories:</p>
<p align="left">Detecting the exchange of a CRL</p>
<p align="left">We only know of one other customer using that ...</p>
</section>
<section data-background="#E6E68A">
<h2>What isn't working?</h2>
</section>
<section data-background="#E6E68A">
<h2>Guess -> Fix is encouraged</h2>
</section>
<section data-background="#E6E68A">
<h2>Predicts incidents or problems can be triaged</h2>
</section>
<section data-background="#E6E68A">
<h2>Focus on the abstract, not the function</h2>
</section>
<section data-background="#E6E68A">
<h2>Falsely assumes predicatbility - especially wrt Capacity</h2>
</section>
<section data-background="#E6E68A">
<h2>Actively discourages direct communication between people</h2>
</section>
<section data-background="#E6E68A">
<h2>Ineffective measures, often set too early to reflect the real world</h2>
</section>
<section data-background="#E6E68A">
<h2>Requires a prediction of what will work in the end ... Places the wrong actors in the feedback loop</h2>
</section>
<section data-background="#E6E68A">
<h2>Fails to rehearse decision making - and understand the fear of exceptional process</h2>
</section>
<section data-background="#E6E68A">
<h2>Misdirects attention to failure rates not repair times</h2>
</section>
<section data-background="#E6E68A">
<h2>YBYRI is part of the answer, but burdened by opportunity and people cost</h2>
</section>
<section data-background="#E6E68A">
<h2>Has controls designed for a world without continuous delivery</h2>
</section>
<section data-background="#A3C2FF">
<h2>What can we do better - in build, transition and operations</h2>
</section>
<section data-background="#A3C2FF">
<h2>Fixing Build</h2>
</section>
<section data-background="#A3C2FF">
<h2>Consider paying for trade-offs in development</h2>
<p align="left">Log-driven development</p>
<p align="left">Intermittently failing system tests - invert the pyramid </p>
<p align="left">Building blocks, not magic boxes</p>
<p align="left">Nygard patterns, Async I/O, Actor Model</p>
</section>
<section data-background="#A3C2FF">
<h2>Institute Leadership</h2>
<blockquote cite="https://en.wikipedia.org/wiki/No_Silver_Bullet">
<p>“Whereas the difference between poor conceptual designs and good ones may lie in the soundness of design method, the difference between good designs and great ones surely does not.</p>
<p>Great designs come from great designers.</p>
<p>... very best designers produce structures that are faster, smaller, simpler, cleaner, and produced with less effort. ”</p>
</blockquote>
<p><small>Fred Brooks, No Silver Bullet, 1986</small></p>
</section>
<section data-background="#A3C2FF">
<h2>Institute Leadership</h2>
<p>Finding the simplest path to most destinations</p>
<img width="700" height="400" data-src="images/ChooseBoringTech.png" alt="Map Problems to Few Solutions" cite="http://mcfunley.com/choose-boring-technology">
</section>
<section data-background="#A3C2FF">
<h2>Design-in A Sanctuary Process</h2>
<img width="800" height="350" data-src="images/datacenter_scale.png" alt="XKCD 1737" cite="https://xkcd.com/1737">
</section>
<section data-background="#A3C2FF">
<h2>Manage a Service Model as an uncertain requirement</h2>
</section>
<section data-background="#A3C2FF">
<h2>Fixing Transition</h2>
</section>
<section data-background="#A3C2FF">
<h2>Monitoring</h2>
<p align="left">Use monitoring continuously</p>
<p align="left">Relatively prioritised user stories to derive screens</p>
<p align="left">Investigations aided by having it all in one place ... forever</p>
<p align="left">Complexity means no a priori knowledge of importance ...</p>
</section>
<section data-background="#A3C2FF">
<h2>Preparing for live</h2>
<p align="left">Look hard for edge cases, smooth them out</p>
<p align="left">Rehearse responding to the unexpected</p>
<p align="left">Prepare for the shock of live ... Plan for how it will impact people</p>
</section>
<section data-background="#A3C2FF">
<h2>Be ready to change</h2>
<p>Continuously improve continuous delivery</p>
</section>
<section data-background="#A3C2FF">
<h2>In Operations</h2>
</section>
<section data-background="#A3C2FF">
<h2>Use the sanctuary process ... even when it isn't necessary ... even when it probably won't work</h2>
</section>
<section data-background="#A3C2FF">
<h2>Stop guessing - Invert Lines, Recover, Five Whys, Be the Bookie </h2>
</section>
<section data-background="#A3C2FF">
<h2>Communicate directly - remove impediments</h2>
</section>
<section data-background="#A3C2FF">
<h2>Audit what happens - detection is a better goal than prevention</h2>
</section>
<section data-background="#A3C2FF">
<h2>Enforce stop and think before action (other than sanctuary process) ... not just stop</h2>
</section>
<section data-background="#A3C2FF">
<h2>Five whys behaviour, not just technology - stupidity is not an explanation</h2>
</section>
<section data-background="#A3C2FF">
<h2>Stop starting and start finishing</h2>
</section>
<section data-background="#A3C2FF">
<h2>Mine the data</h2>
</section>
<section data-background="#A3C2FF">
<h2>Put the team in the chalk circle and let them control the impact</h2>
</section>
<section data-background="#A3C2FF">
<h2>Maintain cleanliness</h2>
</section>
</div>
</div>
<script src="lib/js/head.min.js"></script>
<script src="js/reveal.js"></script>
<script>
// Full list of configuration options available at:
// https://github.com/hakimel/reveal.js#configuration
Reveal.initialize({
controls: true,
progress: true,
history: true,
center: true,
transition: 'slide', // none/fade/slide/convex/concave/zoom
// Optional reveal.js plugins
dependencies: [
{ src: 'lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'plugin/zoom-js/zoom.js', async: true },
{ src: 'plugin/notes/notes.js', async: true }
]
});
</script>
</body>
</html>