Charity Majors+ Your Authors @mipsytipsy CTO @honeycombio; co-wrote Database Reliability Engineering; loves whiskey, rainbows. I test in production and so do you. 🌈🖤Black Lives Matter🖤 May. 29, 2020 1 min read + Your Authors

Years ago, at Linden Lab, we struggled to share tribal knowledge about production, and many engineers were terrified of touching prod (esp new hires)

Enter Shrek Ears. Anyone who broke prod would wear the Ears the rest of the day. I know; sounds awful. ☺️

It wasn't. If people saw you wearing the Ears, they would ask you eagerly for the story -- what happened?! How did you find it, what was the fix? -- and regale you with their own stories.

It was socially leveling -- you'd see the senior engineers sporting Ears in good cheer.

If anyone was self flagellating, we could correct it on the spot and say.. we don't do that here. "You did the reasonable thing! How can we make the system better?"

It became a rite of passage for new hires. "Now you are a blooded warrior, now you are truly one of us."

Every team is different. The Shrek Ears got people talking and sharing wisdom about prod, and gave new folks the courage to be more daring.

If our problem had been recklessness, or if people had been less friendly and encouraging, it could have been incredibly destructive.

(Aw man, does anybody have a picture of the shrek ears?? I can't find any. 😖)

Oh...we also had a proud and storied tradition of solving engineering problems with stuffed animals. The vorpal bunny of oncall.. @bethanye's enormous stuffed trout... etc

You can follow @mipsytipsy.


Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.

Enjoy Threader? Sign up.

Since you’re here...

... we’re asking visitors like you to make a contribution to support this independent project. In these uncertain times, access to information is vital. Threader gets 1,000,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Your financial support will help two developers to keep working on this app. Everyone’s contribution, big or small, is so valuable. Support Threader by becoming premium or by donating on PayPal. Thank you.

Follow Threader