In my experience, very few people understand the techniques, limitations, and implications of encryption (and the attacks upon it). I work with, and have worked with, a large number of talented software engineers and system administrators, and for the most part their understanding of this topic is only a surface understanding. The public image of encryption  and hacking on CSI, and Mission Impossible only makes it worse. I don’t expect people to undestand this technology and its complexities. It’s too hard. Just relax and understand that you have no clue, and most likely no normal manager, politician, or non-specialized technologist does either. It is VERY complex.

What is not complex is the understanding that encryption is important to you. Personally, economically, socially, and politically, knowledge is power and control. We already live in a world where powerful people and organizations are allowed to keep more secrets than individuals. A world without legal strong encryption could easily become one where the powerful have unlimited secrets, and you are allowed zero.

It makes me very glad to see Apple and Tim Cook fighting the police and the President for your right to have some privacy in your life.

I strongly agree with John Gruber’s statement in support of Tim Cook:

TIM COOK LASHES OUT AT WHITE HOUSE OFFICIALS FOR BEING WISHY-WASHY ON ENCRYPTION

Jenna McLaughlin, reporting for The Intercept:

Apple CEO Tim Cook lashed out at the high-level delegation of Obama administration officials who came calling on tech leaders in San Jose last week, criticizing the White House for a lack of leadership and asking the administration to issue a strong public statement defending the use of unbreakable encryption.

The White House should come out and say “no backdoors,” Cook said. That would mean overruling repeated requests from FBI Director James Comey and other administration officials that tech companies build some sort of special access for law enforcement into otherwise unbreakable encryption. Technologists agree that any such measure could be exploited by others.

Nick Heer, at Pixel Envy:

Apple — and Tim Cook, specifically — is the only major tech company currently defending encryption against intrusive surveillance to this degree. Every other company is either open to compromise publicly, has privately compromised, or has failed to take a firm stand.

This came up during last night’s Republican primary debate — not about tech companies refusing to allow backdoors in encryption systems, but about Apple specifically. Tim Cook is right, and encryption and privacy experts are all on his side, but where are the other leaders of major U.S. companies? Where is Larry Page? Satya Nadella? Mark Zuckerberg? Jack Dorsey? I hear crickets chirping.

Real leaders have courage, and on this very essential issue — in the face of fierce political pushback from law enforcement officials — only Tim Cook is showing any.

Thank you.

I’m starting to get the hang of the Docker thing. I’ve been doing it _just barely_ long enough to see some change, and I’ve read material over a longer period. I think it’s important (from an architecture planning perspective) to constantly keep in mind that Docker is a set of very rapidly changing abstractions over a set of solid long-term base functions.  Often the abstractions really suck for a while, since they haven’t truly figured out what things are going to look like in the end.

So we have lots of instances of:

In the past we used this crazy workaround for a feature being missing <——> now we have a half-baked abstraction that may or may not be better than the crazy workaround <——> when we get to use docker 1.9 (or 2.5 or whatever) it will make sense in this way better way.

Logging is one of those that is probably pretty solidly fixed in docker 1.9.

In networking, the new shape in docker 1.9 looks awesome, so by docker 1.11, it should be solid.

Volumes are just starting to get a real picture laid out for us.

So for volumes, we have four ways that docker manages volumes:

1. Host mount (i.e. -v /home/josh/folder:/dock/app/folder): this completely makes sense, and is solid. The only problem with it is that it doesn’t allow any sort of clustering tools to manage/move the containers around, since the volume is outside of the management scope of docker. It’s a host volume, so it’s tied to the host. In the tight scope of our needs for doing Postgres, I think this is the option we should stick with. It just makes things explicit, and we don’t need to move our containers.
2. Anonymous volumes: (i.e. -v /dock/app/folder or VOLUME /dock/app/folder in dockerfile). This creates a randomly named folder somewhere in /var that is mounted into the container. If you don’t docker rm –v the container when you are done with it, this folder will get orphaned, and you’ll leak disk space. While you can use docker inspect to find the folder, there is no real tooling for working with the volume or managing it’s lifecycle. By themselves, these volumes aren’t really useful. They have some performance and reliability implications, but really they are more of a hole in the abstraction, than an operation tool in their own right.
3. Volumes-from: Using #2, we can create a container that has anonymous volumes, but give them a name/handle because they have a container that they are connected to. That container shouldn’t even be running, it should have been run and stopped or just by using the docker create command. Running the second container with —volumes-fom will have the storage from the first container be used as the persistent location for files from the running container. This idea is more compatible with docker cluster managers and portability, but it’s also awkward. You can’t create a data container with Docker Compose. You end up with IMPORTANT stopped containers lying around on your host (a lot of scripts that are out there for cleaning up orphans just delete all of your stopped containers…). For a long while, this has looked like the ‘docker way’ of managing persistent storage, but now it appears that things are changing…
4. Named volumes (i.e. -v myvolume:/dock/app/folder) in Docker 1.9, a cluster of new ‘docker volume’ commands showed up, and the ability to give an ‘anonymous volume’ a name showed up, and volume drivers are in there too. I wouldn’t dream of relying on this functionality yet, as it seems like things are rapidly changing, and theres a pile of bugs and feature requests on github on the subject, but this is a clear abstraction and vision for the future of volume management. In the future, there will be some sort of separate storage server, with it’s own management tools. You’ll be able to cluster and cache and backup volumes through that tool (probably there will be a number of competing solutions), and you will just run your container with it’s volumes specified by name and connect to the volume driver and it will be automagically managed for you. If you need to move containers around, the volume storage server will make sure your data moves too. We should absolutely use this. Next year or sometime.

TLDR: let’s just use host volumes.

In coming up with a solution, it’s good to have a checklist to work from. You don’t have to check all the boxes, but it’s useful to know which ones you’re taking a miss on. Anatomy of a Modern Production Stack · 80%

I’m reading Git From The Inside Out, which is an excellent tutorial on Git, from the viewpoint of it’s technology and data structures (rather than just a quick run through of the basic commands), so I thought it might be a good time to list out some other resources on using Git. Firstly the live tutorial by GitHub and Code School and the git book are worth looking into.

For graphical git clients on the Mac, the new hotness is GitUp, Atlassian gives out SourceTree for free, and GitBox or Tower are popular with the community. Working Copy is a git client for iOS so that you can read and work your code while on the go.

If you are in a dark place, How to undo (almost) anything with Git and On undoing, fixing, or removing commits in git provide some guidance on bailing yourself (also available in flowchart form).

For a little fun, GitLab posted a profile of the 7 Git Personalities. They just need a FaceBook survey so that we can all post our identity. There are a lot of ways that people use git in their workflow and handling the basics of branching. One is GitFlow…obligatory GitFlow Considered Harmful. If you’re feeling the git pain, maybe Advantages of Monolithic Version Control will make you feel less alone.

Beyond the cycle of pain of hostile interviews and the transition to rational interviews, the next evolution has to be about creating a company that people want to work at and brag about. We should have developers knocking at the door and peeking in the windows, wanting to work at such a place. But how?

As a passionate, involved worker, there is too much that I want from an employer. How can any organization be excellent at all of the things that I need and allow me to really achieve? Look at the list of things you need as a software developer and that an employer needs to provide to be a place that will be fulfilling:

• Productive tasks
• Healthy customers
• A work space that is collaborative but not distracting
• Creative ideas and vision
• Engaged coworkers
• A set of operable work practices
• A motivating mission that keeps you inspired
• A technical and business infrastructure
• Supportive mentors that care about your growth
• Opportunities for self-actualization and growth over an extended period of time
• A community of humans that you can be welcomed in
• Patience, care, and financial support of your health

It seems obvious that all of that is way too much to expect from a single organization, and yet with the structure of the full time job in the USA, we are only allowed to select one organization to provide the great majority of these needs.

If you want to run a good company and both hire and hold on to good employees, you need to have answers to all of these needs. Somewhere in the process of attracting and hiring new employees, these things need to be communicated, or you will have the same problem as every other business on your street and in your city: finding ‘qualified’ people.

Good Managers

A big component here has to be good managers. Software engineers have widely varying skill levels when it comes to simple task management and communication. Add cultural elements, and career evolution into the mix, and you’ll very rarely find a good programmer who is a good self-manager. If you further add any real organizational barriers, the job is impossible for even a good self-manager to do himself. Unfortunately, most technical managers are appointed from the outside on the basis of their ability to flog progress on tasks out of an engineering team. Wider goals of cultivating their staff are just not part of the mandate.

When engineering management is done right, you’re focusing on three big things. You’re directly supporting the people on your team; you’re managing execution and coordination across teams; and you’re stepping back to observe and evolve the broader organization and its processes as it grows … There are two things you should always be thinking about: People’s day-to-day and their year-to-year.

Jessica McKellar

A manager needs to be a voice of courage and mission, since not everyone on the team can be brave and focused everyday.

Every committee or organization has at least one well-meaning person who is pushing to make things more average. … And, by amplifying the voice of the lizard brain, he gets under our skin and we back off, at least a little. We make the work a little more average and a little worse.

Seth Godin

There is a failing of traditional engineering management. The best decision making for evolving our products usually occurs with the people doing the actual implementation. They have a commitment and insight and feel for their choices that a ‘boss’ cannot hope to replicate. At the same time, there are a number of duties that a team needs performed that are necessarily not the first focus of software engineers such as communicating clearly and redundantly with other groups, organizing the high-level roadmaps, being a coach/counselor/mentor, creating a framework for enlightened hiring, and more.

The typical authority-based manager is incentivized to ‘boss’ the decisions, despite being imperfectly informed, while leaving the nuts and bolts of facilitating the team and culture to his subordinates, since those tasks are beneath his dignity.

Our managers are not our parents. Their job isn’t to tell us to clean our room and eat our spinach. Our managers should be our coaches, our gurus, our fixers. Old school ‘adults in the room’ managers won’t build the teams that attract and hold great engineers.

The world of “management” is vertical. Its natural habitat comprises tall buildings in places like New York. Its mindset is also vertical. “Strategy gets set at the top,” as Gary Hamel often explains. “Power trickles down. Big leaders appoint little leaders. Individuals compete for promotion. Compensation correlates with rank. Tasks are assigned. Managers assess performance. Rules tightly circumscribe discretion.” The purpose of this vertical world is self-evident: to make money for the shareholders, including the top executives. Its communications are top-down. Its values are efficiency and predictability. The key to succeeding in this world is tight control. Its dynamic is conservative: to preserve the gains of the past. Its workforce is dispirited. It has a hard time with innovation. Its companies are being systemically disrupted. Its economy—the Traditional Economy—is in decline.

Steve Denning

In many cases, it is easy to see that part of the compensation package for being a manager is the privilege of excercising power over your subordinates, limiting their access to information, and the gratification of being higher paid and having better work amenities than them. An organization that incentivises management in this way, and attracts managers who _want_ this kind of ‘payment’ is going to have a hard time getting value for what they pay their engineers.

“managers are being paid in non-pecuniary status benefits via underpaying their underlings, in a way that reduces total firm output, and there are coordination & principal/agent problems in getting management to agree not to be compensated in this way”.

flatmoney  (hacker news)

Just because managers are often poor doesn’t mean they aren’t required, though. Information sharing, goal setting, and power inequalities are facts of business life, if not human life. Someone has to be responsible for negoatiating those waters. If you don’t name that person, you haven’t elminated the focus of power, you’ve just made it harder to figure out who holds that focus.

“Flat” organizations are often alluring lies that hide the real power structures. It doesn’t matter if Bob is an idiot who works 15 minutes per week; if the CEO will take Bob’s word over yours when it comes to your value to the company, then Bob is your boss. … why people hate management: we’ve all picked up that it’s serving its own interests rather than that of the employees or of the company. All of this said, “no management” tends either to produce an emergent and less accountable management/police force or it tends to mean that power is concentrated at the top.

Michael O. Church

Another essential role for managers in an engineering team is to optimally take advantage of the specific genius of the team members, while compensating for the ways in which those same employees are disruptive to an egalitarian team. Engineer types such as the Free Electron or the Honey Badger need specific accomidation. The team needs negotiation between Stables and Volitiles. Extroverts and socially adept team members provide a lot of obvious value, and so a manager needs to exist to elevate the introverts.

Dealing with all of these different types of people while continuously, conscientiously trying to achieve the best outcomes for them and the company is a real job, and it needs talented, dedicated people. Creating or finding these people is way beyond the scope of this post, but 10x developers and 10x teams need this kind of leadership, very badly.

When you’re making a critical decision, you have to understand how it’s going to be interpreted from all points of view. Not just your point of view and not just the person you’re talking to but the people who aren’t in the room, everybody else. In other words, you have to be able, when making critical decisions, to see the decision through the eyes of the company as a whole. You have to add up every employee’s view and then incorporate that into your own view. Otherwise your management decisions are going to have weird side effects and potentially dangerous consequences. It’s a hard thing to do because at the point when you are making a decision, you’re often under a great deal of pressure.

Ben Horowitz

Good Culture

Are you psyched about your company culture the way these guys are?

(The video at http://www.youtube.com/watch?v=tILlZRLhBJE has been removed)

Ableton

Ableton has designed their work around very small teams, focus on quality, focus on self-defined process on each little team. They tried bigger teams and Scrum, but found that it didn’t make the creativity flow. Ableton also has development salons and music salons. Developers are encouraged to train each other and share the technology. At the same time, they are encouraged to work in the business domain–in this case music–and share their creativity there as well. The developers care about the product and the mission and have control over the way things get done.

‘We have so many very smart, creative people here that capturing their ideas and also encouraging their creativity is super important and super valuable and I can’t imagine this not being component of a good development team.’

An empowered, connected culture is hard to keep going, especially as a company grows beyond it’s first twenty-something employees. Jason Evanish blogged recently about the changes that a company undergoes at that divide. These include losing the ability for a leader to have direct contact with all of the employees, that ‘what’s going on’ is no longer universal information, that employees become more political and career oriented, and that culture starts to become solid. He quotes Ben Horowitz: “Perhaps the CEO’s most important operational responsibility is designing and implementing the communication architecture for her company. The architecture might include the organizational design, meetings, processes, email, yammer and even one-on-one meetings with managers and employees. Absent a well-designed communication architecture, information and ideas will stagnate and your company will degenerate into a bad place to work.” (Ben Horowitz).

Oh no. Now I’ve done it. I’ve followed the web and found more from Horowitz:

Let me break it down for you. In good organizations, people can focus on their work and have confidence that if they get their work done, good things will happen for both the company and them personally. It is a true pleasure to work in an organization such as this. Every person can wake up knowing that the work they do will be efficient, effective and make a difference both for the organization and themselves. These things make their jobs both motivating and fulfilling.

In a poor organization, on the other hand, people spend much of their time fighting organizational boundaries, infighting and broken processes. They are not even clear on what their jobs are, so there is no way to know if they are getting the job done or not. In the miracle case that they work ridiculous hours and get the job done, they have no idea what it means for the company or their careers. To make it all much worse and rub salt in the wound, when they finally work up the courage to tell management how fucked up their situation is, management denies there is a problem, then defends the status quo, then ignores the problem.

Definitely go read the awesome rant

Culture requires a firm statement of values from a leader who has thought about things first and made decisions and is continuously holding firm on those decisions. Reed Hastings at Netflix did that and documented the culture he wanted to maintain.

I frequently see CEOs who are clearly winging it. They lack a real agenda. They’re working from slides that were obviously put together an hour before or were recycled from the previous round of VC meetings. Workers notice these things, and if they see a leader who’s not fully prepared and who relies on charm, IQ, and improvisation, it affects how they perform, too. It’s a waste of time to articulate ideas about values and culture if you don’t model and reward behavior that aligns with those goals.

Patty McCord

Personal Growth

You have to be intentional about working career growth into your broader engineering planning and execution of projects coming down the road,” she says. The really difficult thing is that not very many people have a clear sense of what they want from their job, and even when they do, they aren’t forthcoming about it with their managers. Good leaders are experts at surfacing this kind of data and making it actionable. … To develop a long-term relationship with an engineer, you have to learn enough about them to provide a framework to think through their career growth together.

Jessica McKellar

I’m terrible at understanding what my long-term motivations and goals are for working at an organization. I really need someone with a clear vision to tell me what the business wants from me, what success looks like, and to help me develop the tactics that will give me credit and recognition for my contributions.

Building a bureaucracy and elaborate rituals around measuring performance usually doesn’t improve it.

Netflix

I agree with this sentiment, even though I’ve been at more companies where the idea of performance review was completely abdicated. I guess no reviews is better than bad reviews, but it really shouldn’t be structured…it should be organic and constant.

But for each of those “diamond in the rough” hires, there are 9 who are equally cheap but correctly priced (i.e. not very good). In the startup phase, these companies tend to assume that their early technical hires were “desperation hires” and throw them under the bus as soon as they can get “real” engineers, designers, and management. That social-climbing dynamic– constantly seeking “better” (read: more impressive on paper) people than what they have– lasts for years beyond the true startup phase.
…
High-level turnover and constant change of priorities and initiatives means that the lower-level people rarely get much done. They don’t have the runway. Ask a typical four-year veteran of such a company what he’s accomplished, and he’ll tell you all about the work that is no longer used, the project canceled three months before fruition because a new CTO arrived, and the miasma of unglamorous work (i.e. technical integration, maintenance necessary due to turnover) generated by this volatility that, while it might have been necessary to keeping the company afloat, doesn’t show macroscopic velocity. That doesn’t make the case for promotion or advancement. Eventually, the high-power people realize that they can’t get anything done because of all the executive instability, and they leave.

Michael O. Church

Seriously. If you want to learn how to sabotage personal growth for your engineers (and you’re probably doing it), then read the above article. If you want to be a better manager, there’s a million books on the subject, some by people linked to in this post. Even if creating your own system and philosophy of good management is too hard, there’s software available that can do most of that for you.

Good Tasks Worked On Collaboratively

What’s fun about going to work? Getting tasks done. Checking off items on the list is like a drug and it’s essential to having an enjoyable work-day. That’s the simple part. Beyond the basic benefit of having tasks and being allowed to complete them, there are dimensions of ‘helping my team out’, and ‘trusting that others will help me out’, as well as creating and tracking all of those tasks.

Tasks are their own reward, and if you have a healthy relationship between task assignment and completion, you don’t need to demotivate your people with false urgency (although, as discussed there, the completed tasks need to do something and have some value).

Tasks are an opportunity to be creative, and they have nothing to do with putting in your hours or getting done with your work day. Punching a clock is what happens when you have a helpless feeling that your work doesn’t finish or add value. This is the opposite of scoring some tasks.

It is amazing how many of the agile processes have been ‘managed’ into being a never-ending grind, when they are essentially designed to provide fast cycles of completion, value, and accomplish. Agile Process is about spinning a bunch of completable tasks! This is the reward you give the developers for them putting up with your changing business needs.

Good tasks get created through the process of communicating before, during, and after doing the work. Make sure you have done everything possible to allow people to communicate with zero friction while they are working. Slack, Skype, IRC, screen presentation software, are an essential start to this. Video conferencing is becoming easier to implement, and new services are springing up, some completely free. There is pairing software that lets you screen share with two cursors on the screen at the same time and plugins for Visual Studio that let you co-work throughout the whole IDE.

Sure, there are some built-in collaboration features in your bug tracker and source control system, but people are innovating really fast in this collaboration space, and if you’re just using the Out of the Box stuff, you’re behind. Enterprise social networks like Asana or Yammer are available. Tools like Peak monitor all of your cloud utilities for activity, and let the whole team know what you are working on without you having to fill out a single TPS report.

You can make your meetings much better by learning about Productive Meetings. Or if that is too much work, just use software that trains you while you use it. There is a lot of value in the meeting that is unharvested because people won’t create agendas or take minutes. Fix that.

A good meeting should produce a crop of actionable tasks, and you need to track those somewhere, right? Give your people complete freedom on how to track their tasks. Experimenting with different task tracking systems is a personal joy and investment in productivity, and it is constantly evolving. No unified Harrison Bergeron solution can possibly give you the benefits that you get from creative people trying to constantly improve. That said, there’s a lot of options. You can check your task list into source control with the code as a simple text document. You can buy everyone Franklin Planners. Screenful has tons of integrations and pretty graphs. Other tools will structure projects and calendars as well as tasks. It’s all good. Play with the tools, change them up, find things that work and stay out of your way and keep everyone focused. Have fun.

Agile process with backlog groomings and sprints and task boards is there to create the valuable tasks that engineers will get excited about finishing. It was not created, nor should it be used, to micromanage engineers to the point where they can never invent anything. If you need to control your engineers and prevent them from harming you with their creativity…you’ve hired the wrong people (or you are really broken)[or both].

Good Practices

Good practices, including communication and documentation, codestyle and unit tests, work flow and estimation, as well as design and architecture are all very important parts of getting the work done more easily and achieving quality results. As important as that, though, good practices provide an arena for engineers on a team to develop mastery. These are things that you can be incontrovertibly ‘good at’ and feel a sense of pride and motivation about, even when you have technical, political, or market hurdles that are dragging you down.

One excellent area to develop good practices is in remote work. With poor office environments being such a common trial, and the challenges involved in getting any group of more people to be consistently in the same place at the same time with any frequency, the ability to work remotely is potentially a huge benefit. Even if remote work is the exception, rather than the rule, creating the skills and habits needed to enable remote work can really elevate your development practice.

since we went remote, pairing has become really popular. No sharing body odour with your colleague, we just share our screens with a headset on. We learn from each other and challenge each other. We keep each other honest and out of rabbit holes. When things get tricky we grab another pair for a quick swarm. There is banter, it’s fun and there is a strong commitment to quality. We continually question our practices and listen to each others’ frustrations. “Are we really creating something that is really needed or just getting those stories across the board?”

Mingle

Zach Holman talks a little bit about what asynchronous and remote work at GitHub is like. There are some very natural, to engineers, ways in which collaborating through your computer ease the overhead of doing things together. One thing I like in his post is the tantalizing suggestion that communicating in *code* and through *source control* can replace the need for whiteboarding and design meetings. Communicate in the medium that your results will appear in, yeah?

Whichever agile practice you actually follow, it is really worth learning the vocabulary and tools of lean and kanban. Very smart people have thought a lot about how to communicate and structure the tasks of software engineering to reduce the amount of work and the amount of wasted work that is performed. Lean and kanban can be an unending source of new insights on smoothing out your day to day process and improving quality.

Engineers don’t hate process. They hate process that can’t defend itself. … If you want to piss me off, if you want me to hugely discount your value, do this: when I ask you a clarifying question that affects how I will spend my time, my most valuable asset, don’t answer the question. This non-answer is the root cause of an engineer’s hatred of process. A tool that should help bring order to the universe is a blunt instrument that incites rage in the hands of the ignorant.

Rands

Kanban is more than a board with sticky notes on it, and moving cards across a board is completely orthogonal to the ideas of reducing waste and eliminating points of delay (James Shore and Arlo Belshee have a number of insights on this in their Single Piece Flow in Kanban talk). Engineering kanban is also designed to reduce multitasking and get things completed more quickly by giving your workers slack time.

Shorter term process, with real, constant delivery of business value is very useful in getting a group on the same page in terms of what is getting done, who is doing it, and why the current piece is being done first. This is important in a business environment where there’s a natural tendency of management to want to provide a general hand-wave of direction when requesting a project, and then to immediately demand an estimate. With insufficient information and pressure to commit to a timeframe which is inherently unknowable, it is likely that communication with the business will devolve into estimation games that serve to encapsulate the lies of poor estimates in a capsule of emotionality and politics that only serves to establish long-term distrust and disfunction.

So, the shorter term planning process of sprints or cycles or iterations serves to create a level of honesty and trust and communication. This cannot be left alone as the end goal, however. If the end of mature process in an organization is scrum, there will never be a place for discussion of strategy or a place for senior develops to do the research that truly justifies their skills and salary. Eternal scrum can be a demoralizing, micromanaging grind of small tasks that can eventually demoralize your teams and destroy your company.

Just don’t buy too deeply into the idea that by getting the responsibilities of your software right, that you will somehow reduce the impact that all of that business dysfunction has on you as a software developer. Part of the maturation process for a company is cleaning up its’ business processes in parallel to cleaning up its’ software processes.

Udi Dahan

Good Health

Software engineering isn’t a healthy career. I don’t mean to say that it has to make you unhealthy, but rather that the format of the work isn’t inherently one that develops health. Sitting quietly, thinking and typing doesn’t do a lot for your physical health in the way manual labor can. Creating conditions for extreme mental focus can be wearing physically as well as psychologically. Creating an excellent culture around software engineering mean that you have to design health into your culture in ways that are innovative and sometimes risky.

One area of health improvement needs to be a focus on the length and nature of the work day. Lots of overtime and extended periods of crunch time end up being counterproductive and are poor for engineer health in some obvious ways. It takes courage to limit the amount of work time your team puts in, even though the evidence shows that putting in less than eight hour days improves results. Different people have different needs for work hours, and adults can supervise themselves.

The layout of your workplace is another factor in encouraging or destroying employee health. It is well known that a noisy workplace is detrimental to knowledge workers, and yet cubicle farms and open office places are the norm. I’m sure you work with some people who are introverted, since software engineering is a career that is made up of a large proportion of introverts. The panic of being exposed to lots of people with little privacy can tear up the psyche of an introvert in ways that are impossible to justify on the basis of the marginal cost savings involved in square footage.

One solution to the troubled office environment is remote work. If you have a noisy, crowded, non-private office…at least you can work from home. Stanford research has shown that remote workers are more productive and healthier. Companies like 37signals and Github have nailed down their place as market leaders in the context of aggressive remote work capabilities.

Creativity

The mindset needed to get a large team focused on tasks and meeting timelines for complete features is very important for the success of a business. A focus on tasks and limited scope is also valuable to engineers as it provides a means to have constant success and feedback about work well done.

I want to point out, however, that getting those tasks done reliably and quickly is more than an assembly job. It requires creativity and flow. Having a culture that, like Github, celebrates that required creativity and nurtures it is very important in achieving results out of your teams. It is also, in the long term, essential for keeping that irreplaceable, expensive team together.

Good Coworkers

“I’ve learned that I’d rather work by myself than with subpar performers,”

Netflix Employee

Your company acts as a filter for the people who work there. Do you filter out the bad people and concentrate together a better, higher quality group focused on excellence? Or do you filter out the excellence?

If you’re careful to hire people who will put the company’s interests first, who understand and support the desire for a high-performance workplace, 97% of your employees will do the right thing. Most companies spend endless time and money writing and enforcing HR policies to deal with problems the other 3% might cause. Instead, we tried really hard to not hire those people, and we let them go if it turned out we’d made a hiring mistake.

Netflix

Good Candidates

This has been a (by no means comprehensive) survey of some gripes and opportunities in the area of company culture. Working on this stuff is hard, but if you’re not making a real effort to work in at least some of these areas, you shouldn’t gripe about the fact that you can’t find quality people to fill your empty seats. Quality people have jobs right now, and they don’t have any real motivation to come work for you if your culture shows a half-hearted, hand-waving interest in their well being.

Resources:
http://www.culturecreationcanvas.com/

The .NET world is not evolving at the breakneck speed you find in the browser/Javascript world, but in the last year, we’ve seen a lot of the rust blown off the ecosystem and community. .NET is starting to be an exciting place once again, and with some love, courage, and effort we can bring the excitement of the Ruby/Go/Node/JVM worlds back to .NET and be technology leaders rather than simply Microsoft fanbois. Newness itself is not necessarily a virtue (this is the weakness of the Old and Tired/New Hotness format), but everything here has something interesting to learn even if it is only to improve your use of the Old and Tired frameworks and tools you have. New Hotness isn’t about throwing out your old work, it’s about creating a change in you that creates excitement and freshness in the work you do.

Old and Tired: .NET Framework
New Hotness: Core CLR
The good old .Net framework. So solid and fast and useful. Unfortunately it is wired into the core of Windows, which makes it more of a challenge than it should be to change and update. And try and run it on a non-windows operating system. Not too long ago, serious people were starting to suggest that .Net’s days were numbered because of its closed source slow evolution. Today, we have a fresh shiny implementation that has been cut completely free from Windows and can be evolved with a github pull request. CoreCLR can be deployed in your application folder and runs great on Linux and MacOS. I’m sure we will start seeing it integrated into things like the Unity game engine. Exciting times!

Old and Tired: Visual Studio 2013
New Hotness: Omnisharp
Visual Studio is an incredible IDE, seriously. Try the other ones 🙁 they aren’t even close. It is, however, very heavy when compared to a text editor. Slow to load. Complicated. These things warp your development. Not always in bad ways. But still it’s warping. By adding itellisense to Vim, Emacs, and Sublime Text, Omnisharp overcomes the biggest hurdle to using a lightweight text editor in your .Net workflow.

Old and Tired: SOAP WSDL
New Hotness: Swagger
Interoperability using restful or HTTP/JSON web services is so much lighter and more flexible than the agony of SOAP proxies and cross component dependencies. Unfortunately, documenting those services, or generating the boilerplate clients can be a pain. Swagger is a standard for documenting a service api using JSON and is a big part of the new Azure API Apps service.

Old and Tired: log4net
New Hotness: Serilog
Log4net and nLog allow you to print out log lines to instrument your application. Serilog behaves much the same way, but incorporates an understanding of the structure of the data your are logging allowing you to store your log in a richer data store and perform rich analysis of the logged data in a way that has always been just too painful to parse out of an old-and-tired-log.txt file. Serilog does this without adding programmer complexity when adding logging to your classes. You can also link it up to Seq for centralized log storage and analysis.

Old and Tired: WCF/IIS/Windows Services/COM Servers/Scheduled Tasks
New Hotness: AKKA.NET
Only small applications are written without a separation of backend duties these days. There are notifications to send, APIs to call, reports to collect. The classic .Net programming model only gives us the barest guidance as to how to structure and code these kind of applications. Akka.Net is a port of the Java Akka framework, which is in turn inspired by the Erlang runtime which is known for providing a highly reliable development model for backend services in telephony and message processing. While nothing on .Net or Java can guarantee the robustness of the Erlang runtime, Akka provides a lot of infrastructure and guidance for making your backend code into manageable components with comprehensible life cycles. The creators have also created a training bootcamp for learning the tech.

Old and Tired: nUnit
New Hotness: Fixie
nUnit has a standard lifecycle for a test and a class full of tests. However, it is one-size-fits-all, which means you have to cram your broad shoulders into a too-tight jacket. Fixie is a configurable, convention-based testing framework that allows you to structure your test classes in a way that aligns with the code you are testing. Need data-driven tests? Structure them that way. Need a new database instance for each test? Can do. Need to run code before and after parts of your test cycle? Do it! The correct structure for your test classes can make the difference between useful, flexible tests and brittle fragile ones that break every time there is a change. Learn more from the Pluralsight course. Fixie also allows for a choice of assertion frameworks rather than including a default one, allowing the test grammar to be more expressive.

Old and Tired: Msbuild, Rake
New Hotness: Fake
Build scripts? F#? Let’s do this!

Old and Tired: WebApi
New Hotness: Angular with SignalR
‘REST’-ful endpoints are great for supplying data and services to your front-end app, but there are a lot of timing issues, serialization issues, and boilerplate code to work with. What if you could just pass data and method calls back and forth with the server with a real-time connection between the web server and the Angular application? With SignalR you can.

Old and Tired: Stylecop
New Hotness: Roslyn Code Formatter
Automatically format code to follow coding standards before checking it in.

Old and Tired: Nuget
New Hotness: Paket
Nuget is great, but it doesn’t model transitive dependencies well, it doesn’t support semantic versioning, has assembly binding redirect problems, and doesn’t provide a lot of control when you have multiple versions of a sub-dependency required by your libraries. Paket provides a nuget-compatible solution for greater control of your dependency chain, and allows additional hotnesses, such as referencing single files of a GitHub repository (dapper, anyone?) instead of requiring a package wrapping a dll.

Old and Tired: SQL Server
New Hotness: OrigoDB
Ok. I’m not sure if this one is ready for the ‘hotness’ label. It’s edge still has a little too much blood on it. The idea is, however, really sexy. Imagine storing all of your database data in RAM, and interacting with it using clear CQRS semantics. Add in replication, pluggable disk persistence and serialization, and the flexibility to model graph, relational, document, and key-value data, and you have the start of something hot.

The .Net ecosystem is getting better and richer, and in a couple of weeks, people from around the world will be getting together here in Portland to celebrate this, and share what they are doing. Why not join them at .NET Fringe?

Experiment with and share your C#, VB, and F# code with .NET Fiddle

Did you hear about the company that got amazing, industry-leading efficiency and results from their engineering team, by having them spend 45 minutes meditating, twice every work day, before starting to code? If you haven’t read this one, go do it now! Robert Glass, in Facts and Falicies of Software Engineering, writes “The most important factor in software is not the tools and techniques used by the programmers, but rather the quality of the programmers themselves. … Tools matter. Techniques also matter. Process, yet again matters. But head and shoulders above all those other things that matter are people.” I think most of us know that the 10x developer exists. Some people are simply more productive and capable than others. 10x developers are responsible for the success of new techniques, new tools, new practices. 10x developers make their managers look awesome, and their companies immensely profitable. I like to think that most people can become 10x (or at least 8x or 6x) developers given the right management, mentoring, environment, and motivations. That is not, however, my current point. The point here is that there are very good developers available to hire. People who code are not reproducable, replaceable parts. People who code are not identical or even roughly equivalent. If you can accept this fact, then it follows pretty clearly that a hiring process that is focused on filtering out the lowest-quality candidates, and passing whoever is left is aiming for a very low level of success. A disqualification process can only give you low-average results. What if we had a process that focused on detecting the very best developers and capturing them?

We thought [we needed to have a disqualification process]. But the (nearly) resume-blind process we settled on quickly converged on a nearly all sleeper candidate pipeline; in several years, we hired I think just 2 people that had our field on their resume. … Recruiting is two problems: outreach and qualification. … the changes we made to qualification were critical, instrumental, fundamental to our success. Most of our best hires could not have happened had we qualified candidates the way we did in 2009, and the way most firms do today.

photo credit: Hiring Nerds! via photopin (license)

I’ve been seeing a lot of great discussion on the topic of hiring in software engineering lately, and it’s had my mind spinning and my gut churning. The whole thing is a mess. Some of that mess is just human. Sigh. Humans disappoint me. On the other hand, we can learn, and in so doing make things better a little bit at a time.

One of the reasons that hiring is so frustrating is the cycle of abuse. We are used to companies treating their applicants as adversaries. The company wants you to supplicate yourself to them, begging for their approval to work on their yet-another-enterprise-ball-of-mud-legacy-dig-pit project and prove your passion and commitment to their mission of making quarterly sales goals and not getting middle-managers fired.

In return, they entice you with, um…well, they don’t want to tell you what you will get paid (even though they should) (to be fair, salary isn’t a useful lever for getting better performance either). They don’t have much going on in culture, so they don’t have much to tell you about.

Heck! Why are you applying for this thing anyway? Oh yeah, because you’re desperate. You need ‘a job’ andthis is where you get them. No choice. You’re going to go through this whole painful process, and half the time they will slam the door on your face, never telling you anything useful about what went wrong and crushing the little part of you that was starting to like these people and wanting to see them every day more than your family for years.

If they do let you in, you’ll bear that trial as a mark of pride, and make sure that you don’t allow ‘unworthy’ people join your new ranks. The next applicant better prove herself to you, because you wouldn’t want any rif-raf in here. And the cycle continues.

Taking this pain from the other side, however, I see that TechCrunch recently wrote about ‘Secretly Terrible Engineers‘, mocking the job interview process which requires candidates to prove their basic level of competence. Yet, as someone who has been involved in the interview process, I know that the applicants are there which simply don’t know how to code. At all. They are terrible, horrible, negative contributors which make working at a company bad. They are also concentrated (distilled?) in the stream of candidates spurting through the tech recruitment firms, while the quiet, competent majority sits, shell-shocked from their last trial of new employment, at their desk doing a job they are minimally satisfied by since that is easier than finding you.

An fun anecdote from Hacker News:

I remember, rather vividly, the first time I figured out that an engineer couldn’t program. I was attempting to tell him where a value was being assigned in a program. After failing to do so over email, I went over to his desk and asked him to navigate to the file at issue. He was unable to do so. I told him I would do it, assuming that he was unfamiliar with the directory structure in that part of the program, and opened the file. I then said “So you see, the assignment is made in the fooBar subroutine.” He couldn’t find the fooBar subroutine. I said “It is the third method on your screen.” He couldn’t find it. I said “It is this one, here, which I am pointing to with my finger.” He said “OK, that one. Where is the assignment?”
The fooBar subroutine was one statement long.
A coworker, having overheard the conversation, stopped at my desk later, and explained to me that, if I had pressing engineering issues, coworkers X and Y would be excellent senior systems engineers to address them to, but that Z should be allowed to “continue to devote his full attention to work which he is well-suited for.”
The industry has many destructively untrue beliefs about hiring practices. That there exist at least some people who are not presently capable of doing productive engineering work is not one of these beliefs.

Screening out truly useless people is a requirement of a process, but it’s a step that is made a lot easier by not having a situation where it is the most important or most used part of the process. Better applicants means you can afford a less punative process, and once you are in that place, you can make your screening system less than a interrogation using brain teasers.

Technical interviews often are that bad, too. The argument against coding interviews is a valid one, since not every engineer can succeed at a challenge like that under the pressure of a hostile interview with their livelihood on the line. Good people are valuable and somewhat rare, and we shouldn’t be screening them out by having a hostile interview.

Thomas Ptacek:

I’ve seen it happen repeatedly over my career; people who’d pass interviews and then, over 15-20 minutes of face-to-face discussion a couple weeks on the job, not be able to comprehend the idea that a C pointer needs to point to valid memory.
That’s what’s so mind-blowing to me about our terrible process today. It aggressively tries to find and dispatch unqualified candidates, so much so that it forcefully repels talented people. And it fails miserably at screening out people who can’t code! It is the worst case scenario of processes.
It’s also really important to remember that the problem is People Who Can’t Code, not “People Not Smart Enough To Code”. Time and again I’ve “worked” with people who could very effectively critique my own code, who could participate in design meetings down to the bits-and-bytes layer, and, having taken responsibility for actually implementing some portion of those designs, could not commit usable code to save their life. I don’t know what the deal with these people is, but they are out there: plenty smart, totally fluent, and completely ineffective.
I was talking to Erin this weekend about the phenomenon and realized that these people not only exist, but are actually favored by the way the market works. A smart person can last many months in a dev role before anyone realizes they’re ineffective. Then, it takes many, many more months to manage them out of the team. By the time the story plays itself out, the bad team member has more than a year on the job, and when they parlay that resume line item to another prominent job elsewhere, their previous teammates aren’t outraged but instead relieved. They end up with gold-plated resumes and tons of credibility.

Add into your consideration this: software developers are very poor interviewers. They have a background in the cycle of abuse, so they are likely to perpetuate it. They have interviewed relatively few people in their lives (a shift supervisor at Arby’s probably does more interviewing), so they have a poor baseline to compare from. They have no specific professional training in the management and hiring process. They are likely to base their decision entirely on emotion and hire based on ego or just hire pretty people (beware the pretty people). Worst of all, your developers are expensive, rare resources in your company that should have something better to do than run interviews (and probably would much prefer doing it).

Using software engineers in a context where they are mainly trying to negatively filter a stream of candidates is an expensive, unproductive waste.

Can we make interviews better? I hope to share some thoughts on that in my next post or two. It certainly needs doing, whether you’re the one interviewing or the one being interviewed. Something better is badly needed.