Talent Assessment Dos and Don’ts

Management's hardest task is assessing talent. In general almost everything being done is custom to a certain extent, and it is easy to chalk up success or failure to the team involved and not look at what should have been expected. In sports, there is a concept of WAR (Wins above Replacement), that I think everyone (and I mean every single person with a job)  should use to help understand if a person is beneficial to their org. I have different assessment criteria for each of

• Engineers

• Engineering Managers

• Product Managers

Engineers

I personally think Engineers are the easiest to judge in terms of talent. They have tangible work that you can make some assessment on, and most likely if you can pick out the really great engineers (and you better be able to do that at least), they can help you figure out who doesn't cut it. I think of it as 3 things

Use Code not Tickets

In my experience this is a big sticking point for management. They don't code, they shouldn't be architecting (though often they try/do), and they aren't on the hook for on call. Because of this, they want to see tickets done, as that is their only real control over the situation. But I will tell you something you already know, devs don't care about tickets. Most (but only most) of them care about getting their work done, but I've yet to see a dev who thinks managing Jira is more important than getting code in place to solve problems. Also... They are right. Devs are valuable because they can write code that makes products work. They aren't paid to be a Jira trafficker. Managing work is important, but you should not be judging your devs based on their ability to make your Jira life easier. You should be paying them to get good code/design in place. If you don’t, then you are going to create an org which is essentially a support staff, with a few "heros" who get stuff done because management actually lets them code.

Be Honest

Probably the hardest thing to do in reality, as it is easy to rely a lot on feelings. That isn't a judgement, just a fact. As stated above managers aren't in the fray in terms of doing the actual code, so they have to go with their gut. The problem with gut feelings is they are just that, feelings. Feelings are factual, but not truthful. They let you know when something is in alignment (or not) with your values, but they can be misleading when you just want to know if someone can write code (or for that matter manage teams) effectively. In order to be honest, you have to boil things down to facts, which is how much code is being produced by your dev team, and how much should be expected. That last one sounds like a feeling, but it really isn't. You might not be able to compare timelines between teams, but there is no reason that teams shouldn't have very similar throughput in terms of code. If you have a good CI/CD and a good architect/team lead, then you should be able to have every engineer on your team put in multiple PRs every week. If you can't, but don't think they are underperforming, I'll ask you this question, "Are you being honest?".

Stop Deferring

There is always a reason not to fire. We have a deadline. Well they aren't great, but replacing them could be hard. They are the only SME on this subject, so they are important for now. Like any debt, sometimes you have to hold it in order to make a bigger gain, but make no mistake it is a debt. Every day you have poor performers employed, your high performers notice. And this time I'll tell you something you might not know, it pisses them off to have to work with these people. I have talked with dozens of high impact developers, and the only commonality they had, was their level of annoyance of having to carry other developers. That is why you have to pay down this debt as soon as possible. Even if it will hurt to cut a poor performer who has an area of expertise you need, just wait until a high level dev puts in their 2 weeks because you didn't want to do the tough part of the job.

Engineering Managers

The hardest area to do talent assessment is engineering management. Their tangible work output is almost nothing (not casting shade, just stating a fact), so determining if they helped get something out the door or just existed is really difficult. Another problem I find is the upper management gets enthralled with Managers who “know the tech”. Having been a dev and managed, I can tell you “knowing the tech” generally means they aren’t a great manager. I want my managers to manage, and if they are spending time in the tech, that means they aren’t managing, or hire poorly. Neither of those do I find to be great qualities in a manager. For Engineering Management, I try to look at 3 things

Who was their last fire?

The hardest job for an engineering manager is firing someone. Everything else might be stressful, but nothing is more difficult than to sit across from someone and tell them they aren't performing. This is also why you have to be judging them on their ability to do it. Regardless of how well a manager does the rest of their job, if they allow any dev or manager to remain despite all evidence pointing to their need to get rid of people, then they need to go. If you don't, you will end up with an org devoid of talent, as the most talented don't want to consistently have to make up for the under performers.

Do they give estimates, or do they hit deadlines?

Some engineering managers live under the assumption that as long as they can estimate things correctly, then they are doing their job of getting products out. This is obviously a ridiculous assumption, that can be shattered with a simple question. “If I estimated it would take me 30 weeks to build a simple calculator would you think I was good at delivering products?” To everyone who said no, you are correct. To anyone who said yes, you are wrong. To anyone who said "well it depends", you are probably terrible to work with given your need to debate everything.

Managers should be there as a counter-balance between devs and product. Product wants things done NOW, but is also out of the tech loop, so they have a hard time understanding the implications of what a NOW solution has in terms of cost. Devs want things done RIGHT, but also like to silo themselves away from the practical realities of running a business, where you need to keep adding features/improvements and can’t just focus on how to get something perfect. A manager should blend the right balance, where they are

• Far enough away from the code, to avoid any bias about the maintenance costs, but close enough to the devs to understand when there is a real cost implication of a NOW solution.

• Far enough away from the code, that they aren’t architecting, but close enough to the devs to understand what is a big change and what is not.

• Close enough to the Product, that they can ask questions to nail down the value proposition, but far enough away they don’t suddenly think they are the expert.

• Close enough to the Product, that they can align multiple Product asks and understand overlap, but far enough away they aren’t implicitly favoring 1 part of the product over another.

Are they an Executive or a Manager?

Something I often see managers doing is playing part time therapist, rather than focusing on execution of tasks. You probably see that as well when Managers have 6 directs, but suggest they are busy (doing what?). This happens because if you look at the calendar they are doing a lot of faux coordination and weekly 1 on 1s which really don’t add value. If you have a dev who hasn’t just started or is on a PIP, and they need weekly 1 on 1s, you either aren’t paying enough attention to your team, or that person is a liability and you should be assessing whether they need to stay. You can tell whether these 2 are happening when

• Faux Coordination - Your manager meets with several other managers multiple times a week in order to “stay in sync”. This is busy work, and what you really need is your Staff level devs meeting once a week, excepting for ad-hocs, as they ensure they have the right API needs. Managers talking with Managers very rarely results in anything productive.

• Part Time Therapist - You see this when you have otherwise well adjusted adults meeting with a manager consistently every week. This is basically a trope that has become the norm. When I’m managing, I have no reason to sync with my devs more than every 2 weeks unless they are brand new to the team and have some questions about process/talent evaluation we need to cover. Magically these devs did not become a hindrance, but started noticing they are in too many meetings.

When the above 2 aren’t happening, then you have an Engineering Manager who is acting more like an Executive, which is really what you want. They are there to help improve the execution of tasks/process, not just put process/meetings in place to pretend to be working.

Product Managers

This is an area I have no personal experience with in terms of actually being one, but I can tell you from a dev and management perspective who the people we are the most productive with. Generally they share the same characteristics, even if they bring their own flavor/personality to the work.

• They document everything in incredible detail - A lot of product owners can answer questions as you come to them, but very few can lay out their vision via either docs or figma or some combination of the 2. That takes a level of detail that is hard to find.

• They have a Quarter and an 18 month perspective - In my experience a lot of product managers are just trying to get their next thing out for the quarter. That becomes a glaring issue, when in year 5-7 of a company life cycle it has become impossible to build out features, and the database falls over every other week. A great product manager is trying to get stuff out this quarter, but understands what a true Minimum Viable Product is, and whether to go all in on a product or just be mostly in with eyes towards 12 months from now when the real thing needs to be in place.

• They don’t ask technical details - This is a bit of a cheat, as sometimes it is fun to know details, but for the most part Product shouldn’t care about how stuff is made, only that it fulfills the needs of the user. This allows tech teams to worry only about internal debate and not about also having to make their case to product on why something should be done. If you ever hear anything like “With the old setup we were able to do X/Y/Z”, that person is going to be a pain to work with.

• They work for the company, and not for themselves at your company - This one is hard to quantify, but I bet most everyone reading knows what I mean. The number of times I’ve worked with people who so clearly only cared about doing something that made them look good vs. doing what was actually good for the company is staggering. On an Engineering Team there are enough people where it becomes a little easier to weed those people out, but in product it seems impossible as they are a defacto single point of failure given their Product Knowledge and generally don’t work as a true team.

If you find someone who meets all of those criteria, then you have yourself a steal. If you find someone who doesn’t meet any of those, then I would ask yourself if they are an asset or if you don’t like to have hard discussions about who to hire.